Information processing device and information processing method

ABSTRACT

To improve manipulability. 
     An information processing device includes a wireless communication unit and a control unit. The wireless communication unit performs real-time image transmission with at least one sink device according to a Wi-Fi CERTIFIED Miracast specification. The control unit performs control related to the real-time image transmission based on control information for controlling the information processing device according to the Wi-Fi CERTIFIED Miracast specification, the control information being received from the at least one sink device. The control unit performs control such that on-off switching of a UIBC for transmitting manipulation information in the sink device that has transmitted the control information is performed when a plurality of pieces of control information are received.

TECHNICAL FIELD

The present technology relates to an information processing device.Particularly, the present technology relates to an informationprocessing device, a communication system, an information processingmethod, and a program causing a computer to execute the method in whichvarious kinds of information are exchanged using wireless communication.

BACKGROUND ART

In the related art, wireless communication technologies for exchangingvarious kinds of data using wireless communication are known. Forexample, a wireless communication technology for exchanging variouskinds of information through wireless communication between twoinformation processing devices has been proposed (for example, seePatent Literature 1).

CITATION LIST Patent Literature

Patent Literature 1: JP 2008-278388A

SUMMARY OF INVENTION Technical Problem

According to the above-described technologies of the related art,various kinds of information can be interchanged between two informationprocessing devices through wireless communication even when connectionis not established by wired lines. For example, an image based on imagedata transmitted from a transmission side information processing devicecan be displayed on a display unit of a reception side informationprocessing device.

When the reception side information processing device displays the imagethrough the display unit based on the image data transmitted from thetransmission side information processing device as described above, theuser is considered to perform a manipulation in the transmission side orreception side information processing device. In this regard, it isimportant to improve manipulability of the user.

The present technology was made in light of the foregoing, and it is anobject of the present technology to improve manipulability.

Solution to Problem

The present technology was made to solve the problem, and a first aspectthereof is an information processing device, an information processingmethod thereof, and a program for causing a computer to execute theinformation processing method, the information processing deviceincluding: a wireless communication unit configured to perform real-timeimage transmission with at least one sink device according to a wirelessfidelity (Wi-Fi) CERTIFIED Miracast specification; and a control unitconfigured to perform control related to the real-time imagetransmission based on control information for controlling theinformation processing device according to the Wi-Fi CERTIFIED Miracastspecification, the control information being received from the at leastone sink device. The control unit performs control such that on-offswitching of a user input back channel (UIBC) for transmittingmanipulation information in the sink device that has transmitted thecontrol information is performed when a plurality of pieces of controlinformation are received. Thus, there is an operation in which on-offswitching of a user input back channel (UIBC) for transmittingmanipulation information in the sink device that has transmitted thecontrol information is performed when a plurality of pieces of controlinformation are received.

According to the first aspect, the wireless communication unit mayreceive the control information on a transmission control protocol (TCP)session generated between the information processing device and the sinkdevice. Thus, there is an operation in which the control information isreceived on a transmission control protocol (TCP) session generatedbetween the information processing device and the sink device.

According to the first aspect, the control of the on-off switching maybe control of any one of a connection, a disconnection, a start, a stop,validation, and invalidation. Thus, there is an operation in which thecontrol of the on-off switching is control of any one of a connection, adisconnection, a start, a stop, validation, and invalidation.

According to the first aspect, the control unit may perform control suchthat the UIBC is validated for a first sink device among the at leastone sink device, and the UIBC is invalidated for a second sink device.Thus, there is an operation in which the UIBC is validated for a firstsink device among the at least one sink device, and the UIBC isinvalidated for a second sink device.

According to the first aspect, the wireless communication unit may givea notification indicating that the UIBC is set to be valid to the firstsink device, and may give a notification indicating that the UIBC is setto be invalid to the second sink device. Thus, there is an operation inwhich a notification indicating that the UIBC is set to be valid isgiven to the first sink device, and a notification indicating that theUIBC is set to be invalid is given to the second sink device.

According to the first aspect, the control information may be aGET_PARAMETER Response. The notification may be transmitted asSET_PARAMETER. Thus, there is an operation in which the controlinformation is a GET_PARAMETER Response, and the notification istransmitted as SET_PARAMETER.

According to the first aspect, the control unit may perform control suchthat the UIBC is validated for a first sink device among the at leastone sink device, and the UIBC is invalidated for all the sink devicesexcept the first sink device among the sink devices that havetransmitted the control information. Thus, there is an operation inwhich the UIBC is validated for a first sink device among the at leastone sink device, and the UIBC is invalidated for all the sink devicesexcept the first sink device among the sink devices that havetransmitted the control information.

According to the first aspect, the wireless communication unit mayreceive a plurality of pieces of manipulation information associatedwith ID information as UIBC information. The control unit may identifycorresponding manipulation information with reference to an ID headerincluded in the ID information. Thus, there is an operation in which aplurality of pieces of manipulation information associated with IDinformation are received as UIBC information, and correspondingmanipulation information is identified with reference to an ID headerincluded in the ID information.

According to the first aspect, the at least one sink device may includea plurality of input devices. The plurality of pieces of manipulationinformation may be transmitted from the plurality of input devices viathe at least one sink device. Thus, there is an operation in which theplurality of pieces of manipulation information are transmitted from theplurality of input devices via the at least one sink device.

According to the first aspect, the wireless communication unit may setone port for a bundle of the plurality of pieces of UIBC information,and may perform communication. Thus, there is an operation in which oneport is set for a bundle of the plurality of pieces of UIBC information,and communication is performed.

According to the first aspect, the wireless communication unit mayperform communication as a source device. Thus, there is an operation inwhich communication as a source device is performed.

A second aspect of the present technology is an information processingdevice, an information processing method thereof, and a program forcausing a computer to execute the information processing method, theinformation processing device including: a wireless communication unitconfigured to perform real-time image transmission with at least onesource device according to a wireless fidelity (Wi-Fi) CERTIFIEDMiracast specification; and a display unit configured to display animage received through the real-time image transmission. The wirelesscommunication unit receives control information related to turning-on oroff of a user input back channel (UIBC) for transmitting manipulationinformation in the information processing device among at least one sinkdevice from a first source device when the first source device among theat least one source device performs the real-time image transmissionwith the at least one sink device. Thus, there is an operation in whichcontrol information related to turning-on or off of a user input backchannel (UIBC) for transmitting manipulation information in theinformation processing device among at least one sink device is receivedfrom a first source device when the first source device performs thereal-time image transmission with the at least one sink device.

Advantageous Effects of Invention

According to the present technology, an excellent effect that thatmanipulability can be improved can be obtained. The effect describedherein is not necessarily limited, and any effect described in thepresent disclosure may be included.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a system configuration example ofa communication system 100 according to a first embodiment of thepresent technology.

FIG. 2 is a block diagram illustrating a functional configurationexample of an information processing device 200 according to the firstembodiment of the present technology.

FIG. 3 is a block diagram illustrating a functional configurationexample of an information processing device 300 according to the firstembodiment of the present technology.

FIG. 4 is a diagram schematically illustrating a content exampleretained in a management information retention unit 390 according to thefirst embodiment of the present technology.

FIG. 5 is a diagram illustrating a transition example of imagesdisplayed on a display unit 351 of the information processing device 300according to the first embodiment of the present technology.

FIG. 6 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology.

FIG. 7 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology.

FIG. 8 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology.

FIG. 9 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a mode request according to the firstembodiment of the present technology.

FIG. 10 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a mode request according to the firstembodiment of the present technology.

FIG. 11 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a UIBC according to the firstembodiment of the present technology.

FIG. 12 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a UIBC according to the firstembodiment of the present technology.

FIG. 13 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a UIBC according to the firstembodiment of the present technology.

FIG. 14 is a diagram illustrating a configuration example when a sinkdevice performs on-off control of a UIBC according to the firstembodiment of the present technology.

FIG. 15 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology.

FIG. 16 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology.

FIG. 17 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 18 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 19 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 20 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 21 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 22 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 23 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 24 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 25 illustrates an example of a frame format of a UIBC interchangedbetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 26 is a sequence chart illustrating a communication process examplebetween a source device 410 and a sink device 420 according to the firstembodiment of the present technology.

FIG. 27 is a flowchart illustrating an example of a processing procedureof a port setting process performed by an information processing device300 according to the first embodiment of the present technology.

FIG. 28 is a diagram illustrating a communication example of image dataor audio data using a WebSocket between a source device 410 and a sinkdevice 420 according to the first embodiment of the present technology.

FIG. 29 is a diagram illustrating a system configuration example of acommunication system 600 according to the second embodiment of thepresent technology.

FIG. 30 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the second embodiment of the present technology.

FIG. 31 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the second embodiment of the present technology.

FIG. 32 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the second embodiment of the present technology.

FIG. 33 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the third embodiment of the present technology.

FIG. 34 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the third embodiment of the present technology.

FIG. 35 is a diagram illustrating a system configuration example of acommunication system 700 according to the fourth embodiment of thepresent technology.

FIG. 36 is a flowchart illustrating an example of a processing procedureof a non-display region notification process performed by a receiver 710(a source device) according to a fourth embodiment of the presenttechnology.

FIG. 37 is a block diagram illustrating an example of a schematicconfiguration of a smartphone.

FIG. 38 is a block diagram illustrating an example of a schematicconfiguration of a car navigation device.

DESCRIPTION OF EMBODIMENT(S)

Hereinafter, modes (hereinafter, “embodiments”) for carrying out thepresent technology will be described. The description will proceed inthe following order:

1. First embodiment (example in which UIBC and WebSocket are set betweensource device and sink device)

2. Second embodiment (example in which connection between source deviceand sink device via access point and direct connection between sourcedevice and sink device are switched) (or simultaneous used))

3. Third embodiment (example in which source device is connected to sinkdevice or access point through manipulation of user or operation ofuser)

4. Fourth embodiment (example in which one source device is connected toplurality of sink devices)

5. Application examples

1. FIRST EMBODIMENT Configuration Example of Communication System

FIG. 1 is a block diagram illustrating a system configuration example ofa communication system 100 according to the first embodiment of thepresent technology. FIG. 1 illustrates an example of a communicationsystem capable of performing wireless connection through peer to peer(P2P) direct communication.

The communication system 100 includes information processing devices200, 300, and 400. The communication system 100 is a communicationsystem in which the information processing device 300 receives data (forexample, image data or audio data) transmitted from at least one of theinformation processing devices 200 and 400.

The information processing devices 200, 300, and 400 are transmissionand reception devices that have a wireless communication function. Theinformation processing devices 200, 300, and 400 are, for example,display devices (for example, personal computers) or portableinformation processing devices (for example, smartphones or tabletterminals) that have a wireless communication function. The informationprocessing devices 200, 300, and 400 are, for example, wirelesscommunication devices that conform to Institute of Electrical andElectronics Engineers (IEEE) 802.11, 802.15, or 802.16, 3rd GenerationPartnership Project (3GPP) specification (for example, Wideband CodeDivision Multiple Access (W-CDMA)), Global system for MobileCommunications (GSM: registered trademark), Worldwide Interoperabilityfor Microwave Access (WiMAX), WiMAX2, Long Term Evolution (LTE), LIE-A(Advanced), or the like. The information processing devices 200, 300,and 400 can interchange various kinds of information using the wirelesscommunication function.

Here, an example of a case in which wireless communication usingwireless Local Area Network (LAN) is performed between the informationprocessing devices 200 and 300 or between the information processingdevices 400 and 300 will be described.

As the wireless LAN, for example, Wireless Fidelity (Wi-Fi) Direct,Tunneled Direct Link Setup (TDLS), an ad-hoc network, or a mesh networkcan be used. As short-range wireless audio visual (AV) transmissioncommunication used in the communication system 100, for example, Wi-FiCertified Miracast (technical specification title: Wi-Fi Display) can beused. Wi-Fi Certified Miracast is a mirroring technology fortransmitting an audio or a display image reproduced with one terminal toanother terminal using the technology of Wi-Fi Direct or TDLS andoutputting the audio or image data similarly with the other terminal.

In Wi-Fi Certified Miracast, user Input Back Channel (UIBC) is realizedon Transmission Control Protocol/Internet Protocol (TCP/IP). UIBC is atechnology for transmitting manipulation information of an input devicesuch as a mouse or a keyboard from one terminal to another terminal.Instead of Wi-Fi Certified Miracast, another remote desktop software(for example, Virtual Network Computing (VNC)) may be applied.

Here, in Wi-Fi Certified Miracast, for example, it is established thatan image (video) is compressed and decompressed using H.264. Forexample, in Wi-Fi Certified Miracast, H.264 can be adjusted on atransmission side. An embodiment of the present technology is notlimited to H.264, but can also correspond to various codecs such asH.265 (for example, high efficiency video coding (HEVC) and scalablevideo coding extensions of high efficiency video coding (SHVC)) andMoving Picture Experts Group (MPEG4), Joint 1Photographic Experts Group(JPEG) 2000. Further, it can also correspond to a line-based codec inwhich one or more lines are bundled and compressed or two or more linesare divided into 2×2 or more macro blocks to be compressed anddecompressed. For example, by obtaining a difference with a previouscode amount region of a specific code amount region (such as a picture,a bundle of a plurality of lines, or a macro block), it is possible tocorrespond to a codec that reduces a transmission rate withoutperforming compression such as DCT or Wavelet. Further, an image (video)may be transmitted or received with non-compression.

In the first embodiment of the present technology, an example in whichthe information processing device 200 sets image data and audio datagenerated through an imaging operation as a transmission target will bedescribed. In the first embodiment of the present technology, an examplein which the information processing device 400 sets content (forexample, content formed by image data and audio data) stored in astorage unit (for example, a hard disk) as a transmission target. Anelectronic device (for example, a PC, a game device, a smartphone, or atablet terminal) on which a camera is mounted as the informationprocessing device 200 may be used. Another electronic device (forexample, an imaging device, a game device, a smartphone, or a tabletterminal) that includes a display unit as the information processingdevice 300 may be used. If the information processing device 400 has atethering function, the information processing device 400 may acquirecontent stored in an internet services provider (IPS) via wireless orwired network and set the content as a transmission target.

For example, image data generated through an imaging operation of theinformation processing device 200 is transmitted to the informationprocessing device 300 and an image 11 based on the image data isdisplayed on a display unit 351 of the information processing device300. Further, content stored in a storage unit (for example, a harddisk) of the information processing device 400 is transmitted to theinformation processing device 300 and an image 12 based on this contentis displayed on the display unit 351 of the information processingdevice 300.

In this way, in the first embodiment of the present technology, anexample in which the information processing devices 200 and 400 serve assource side information processing devices (source devices) and theinformation processing device 300 serves as a sink side informationprocessing device (sink device) will be described.

In FIG. 1, a range in which the information processing device 300 canperform direct communication via peer to peer (P2P) direct connectionusing wireless communication is indicated as an information transferrange 101. The information transfer range 101 is an information transferrange (a service range) based on the information processing device 300.

[Configuration Example of Information Processing Device (Source Device)]

FIG. 2 is a block diagram illustrating a functional configurationexample of the information processing device 200 according to the firstembodiment of the present technology. The functional configuration ofthe information processing device 400 related to wireless communicationis substantially the same as that of the information processing device200. Therefore, in the first embodiment of the present technology, onlythe information processing device 200 will be described and thedescription of the information processing device 400 will be omitted.

The information processing device 200 includes an antenna 210, awireless communication unit 220, a control signal reception unit 230, acontrol unit 240, an image and audio signal generation unit 250, animage and audio compression unit 260, and a stream transmission unit270.

The wireless communication unit 220 transmits and receives each piece ofinformation (for example, image data and audio data) to and from anotherinformation processing device (for example, the information processingdevice 300) via the antenna 210 using wireless communication under thecontrol of the control unit 240. For example, when an image datatransmission process is performed, the image data generated by the imageand audio signal generation unit 250 is compressed by the image andaudio compression unit 260 and the compressed image data (image stream)is transmitted from the antenna 210 via the wireless communication unit220.

The wireless communication unit 220 is assumed to be able to transmitand receive each piece of information to and from another informationprocessing device (for example, the information processing device 300)using a plurality of frequency channels. In the first embodiment of thepresent technology, an example in which the wireless communication unit220 has a function of transmitting and receiving three kinds offrequency channels, 2.4 GHz, 5 GHz, and 60 GHz will be described. Inthis way, when the source device has the function of transmitting andreceiving the plurality of frequency channels, a sink device (forexample, the information processing device 300) can control a frequencychannel to be used by each source device.

The control signal reception unit 230 acquires a control signal (forexample, information interchanged with the information processing device300) transmitted from another information processing device (forexample, the information processing device 300) among the pieces ofinformation received by the wireless communication unit 220. Then thecontrol signal reception unit 230 outputs the acquired control signal tothe control unit 240.

The control unit 240 performs control on each piece of information to betransmitted from the information processing device 200. For example, thecontrol unit 240 performs control on the image and audio signalgeneration unit 250 and the image and audio compression unit 260 basedon the control signal received by the control signal reception unit 230.For example, the control unit 240 performs control such that the numberof channels of audio or the resolution of image data which is atransmission target is changed or performs control such that an imageregion of the image data which is a transmission target is changed. Thatis, the control unit 240 performs transmission control of a stream whichis a transmission target based on the control signal received by thecontrol signal reception unit 230. The transmission control of a streamis, for example, data transmission speed control, scalabilitytransmission rate control.

The control unit 240 may have a function of measuring a radio wavepropagation situation (link radio wave propagation situation) when datais transmitted to and received from the sink device using the wirelesscommunication and may transmit a measurement result (radio wavepropagation measurement information) to the sink device.

Here, the radio wave propagation measurement information is, forexample, information used to determine whether line quality with thesink device is quality with which the image data and the audio data canbe transmitted and received. The radio wave propagation measurementinformation is used, for example, when stream transmission control isperformed. The radio wave propagation measurement information will bedescribed in detail with reference to FIG. 4. Instead of the radio wavepropagation measurement information, the control unit 240 may count thenumber of retransmissions of the same packet and perform the streamtransmission control corresponding to the counted number ofretransmissions.

Here, the data transmission speed mainly means an occupancy ratio to acommunication line and is assumed to include a meaning of acommunication speed or a communication capacity. For example, theresolution is defined as an index of image quality configured to includea component such as an image frame (the number of vertical andhorizontal pixels) of the image data, or a bit rate (compression ratio)of the image data. As the index of the quality, the throughput of astream can be used. The number of channels of audio is assumed toinclude a meaning of an audio recording and reproducing method such as amonaural (1.0 ch), a stereo (2.0 ch), 5.1 ch, 9.1 ch, or high-resolutionaudio. The number of channels of audio is defined as an index of audioquality configured to include a component such as a bit rate(compression ratio) of audio data or the number of channels. As theindex of the audio quality, the throughput of a stream can be used.

The control unit 240 performs control such that a state unstable in thedata rate control is improved. For example, the control unit 240comprehends system performance information of a sink device (forexample, the information processing device 300) by interchanginginformation with the sink device. Here, the system performanceinformation is, for example, performance information regarding thesystem of the sink device. For example, the system performanceinformation is a usable frequency channel, a resolution, TransmissionControl Protocol (TCP), and User Datagram Protocol (UDP). The systemperformance information is, for example, information indicating each ofcorrespondence of an encryption method, correspondence of standarddefinition (SD)/high definition (HD), and correspondence of a low powerconsumption mode. For example, the control unit 240 can select a methodfor the stream transmission control to further improve the entire systemstability of the communication system 100 according to whether the sinkdevice corresponds to the lower power consumption mode.

For example, the control unit 240 is assumed to insert informationregarding whether the information processing device 200 is a mobiledevice during interchange of information with the information processingdevice 300. For example, capability information regarding theinformation processing device 200 can include information regardingwhether the information processing device 200 is a mobile device. Whenit is comprehended that the information processing device 200 is themobile device, the information processing device 300 can determine thatit is not necessary to operate the information processing device 200based on association with other connected information processingdevices. In this way, when it is determined that it is not necessary tooperate the information processing device 200, the informationprocessing device 200 receives a transmission stop command from theinformation processing device 300. When the control unit 240 comprehendsthe transmission stop command, the control unit 240 can be powered downthe function of each of the image and audio signal generation unit 250,the image and audio compression unit 260, and the stream transmissionunit 270 for a given time. The control unit 240 can transition thewireless communication unit 220 to intermittent reception (which is amode in which the wireless communication unit 220 rises up periodicallyso that the wireless communication unit 220 can receive a command fromthe information processing device 300 and the device is powered down inother cases).

The image and audio signal generation unit 250 generates data (imagedata and audio data) which is an output target under the control of thecontrol unit 240 and outputs the generated data to the image and audiocompression unit 260. For example, the image and audio signal generationunit 250 includes an imaging unit (not illustrated) and an audioacquisition unit (not illustrated). The imaging unit (for example, alens, an image sensor, or a signal processing circuit) images a subjectand generates an image (image data). The audio acquisition unit (forexample, a microphone) acquires a surrounding audio when the image datais generated. The data generated in this way is a transmission target tobe transmitted to another information processing device (for example,the information processing device 300).

The image and audio compression unit 260 compresses (encodes) the data(the image data and the audio data) generated by the image and audiosignal generation unit 250 under the control of the control unit 240.Then, the image and audio compression unit 260 outputs the compresseddata (the image data and the audio data) to the stream transmission unit270. The image and audio compression unit 260 may be realized byperforming the encoding by software or may be realized by performing theencoding by hardware. The image and audio compression unit 260 isassumed to function as a codec, but is assumed to be able to handle anuncompressed image or audio. Further, the image and audio compressionunit 260 can also function as a scalable codec. Here, the scalable codecmeans, for example, a codec which can be applied freely according to theresolution of a reception side information processing device (sinkdevice), a network environment, or the like.

The stream transmission unit 270 performs a transmission process oftransmitting the data (the image data and the audio data) compressed bythe image and audio compression unit 260 as a stream from the antenna210 via the wireless communication unit 220 under the control of thecontrol unit 240.

The information processing device 200 can include a display unit, anaudio output unit, and a manipulation reception unit in addition to theabove-described units, but these units are not illustrated in FIG. 2.The example in which the information processing device 200 generates theimage data and the audio data which are the transmission targets hasbeen described. However, the information processing device 200 mayacquire image data and audio data which are transmission targets from anexternal device. For example, the information processing device 200 mayacquire image data and audio data which are transmission targets from aweb camera equipped with a microphone. The information processing device200 may set content (for example, content formed by image data and audiodata) stored in a storage device (for example, a hard disk) as atransmission target irrespective of the inside or outside of theinformation processing device 200. In this case, the content stored inthe storage device is also assumed to be compressed content. In thiscase, when the compressed content is compressed in accordance with anencoding scheme defined in a standard adopted in the communicationsystem 100, the compressed content may be transmitted without beingdecrypted (decoded).

A display unit (not illustrated) of the information processing device200 is, for example, a display unit that displays an image generated bythe image and audio signal generation unit 250. As the display unit,various display panels can be used. For example, a display panel such asan electro-luminescence (EL) or crystal light-emitting diode (LED)display or a liquid crystal display (LCD) can be used.

An audio output unit (not illustrated) of the information processingdevice 200 is, for example, an audio output unit (for example, aspeaker) that outputs an audio generated by the image and audio signalgeneration unit 250. An image can be output from both of a transmissiondevice and a reception device, but an audio is preferably output fromonly one of the transmission device and the reception device.

A manipulation reception unit (not illustrated) of the informationprocessing device 200 is a manipulation reception unit that receives amanipulation input performed by a user and is, for example, a keyboard,a mouse, a game pad, a touch panel, a camera, or a microphone. Themanipulation reception unit and the display unit can be integrallyconfigured using a touch panel capable of performing a manipulationinput when the user touches or approaches a display surface with his orher finger.

[Configuration Example of Information Processing Device (ReceptionSide)]

FIG. 3 is a block diagram illustrating a functional configurationexample of the information processing device 300 according to the firstembodiment of the present technology.

The information processing device 300 includes an antenna 310, awireless communication unit 320, a stream reception unit 330, an imageand audio decompression unit 340, an image and audio output unit 350, auser information acquisition unit 360, a control unit 370, a controlsignal transmission unit 380, and a management information retentionunit 390.

The wireless communication unit 320 transmits and receives each piece ofinformation (for example, image data and audio data) to and from anotherinformation processing device (for example, the information processingdevice 200) via the antenna 310 using wireless communication under thecontrol of the control unit 370. For example, when an image datareception process is performed, the image data received by the antenna310 is decompressed (decoded) by the image and audio decompression unit340 via the wireless communication unit 320 and the stream receptionunit 330. Then, the decompressed image data is supplied to the image andaudio output unit 350 and an image according to the decompressed imagedata is output from the image and audio output unit 350. That is, theimage according to the decompressed image data is displayed on a displayunit 351.

The wireless communication unit 320 is assumed to be able to transmitand receive each piece of information to and from another informationprocessing device (for example, the information processing device 200)using a plurality of frequency channels. In the first embodiment of thepresent technology, an example in which the wireless communication unit320 has a function of transmitting and receiving three kinds offrequency channels, 2.4 GHz, 5 GHz, and 60 GHz will be described. Thatis, the wireless communication unit 320 can perform communication usinga first frequency band and communication using a second frequency bandof a higher data transmission speed than the first frequency band. Thecontrol unit 370 controls a frequency channel to be used among aplurality of frequency channels in wireless communication with eachsource device.

Link between the information processing devices 200 and 300 and linkbetween the information processing devices 400 and 300 may beestablished with the same frequency channel or may be established withdifferent frequency channels.

In the first embodiment of the present technology, an example in whichthe wireless communication unit 320 has the function of transmitting andreceiving three kinds of frequency channels, 2.4 GHz, 5 GHz, and 60 GHzwill be described, but an embodiment of the present technology is notlimited thereto. For example, the wireless communication unit 320 mayhave a function of transmitting and receiving other frequency channels,two frequency channels, four or more frequency channels.

The stream reception unit 330 receives streams (for example, an imagestream and an audio stream) and interchange information with each sourcedevice among the pieces of information received by the wirelesscommunication unit 320 under the control of the control unit 370. Then,the stream reception unit 330 outputs the received command informationto the control unit 370 and outputs the received streams to the imageand audio decompression unit 340 and the control unit 370.

Here, the interchange information with each source device is informationtransmitted from a source device (for example, the informationprocessing device 200) and includes, for example, a request foracquiring system performance information of the information processingdevice 300. The system performance information is, for example,information indicating a usable frequency channel, a resolution, TCP,and UDP or each of correspondence of an encryption method,correspondence of SD/HD, and correspondence of a low power consumptionmode.

The stream reception unit 330 has a function of measuring a radio wavepropagation situation (link radio wave propagation situation) when datais transmitted to and received from a sink device using the wirelesscommunication. The stream reception unit 330 outputs a measurementresult (radio wave propagation measurement information) to the controlunit 370. The radio wave propagation measurement information will bedescribed in detail with reference to FIG. 4.

The image and audio decompression unit 340 decompresses (decodes) thestreams (image data and the audio data) transmitted from anotherinformation processing device (for example, the information processingdevice 200) under the control of the control unit 370. Then, the imageand audio decompression unit 340 outputs the decompressed data (theimage data and the audio data) to the image and audio output unit 350.The image and audio decompression unit 340 may be realized by performingthe decoding by software or may be realized by performing the decodingby hardware. The image and audio decompression unit 340 is assumed tofunction as a codec, but is assumed to be able to handle an uncompressedimage or audio. Further, the image and audio decompression unit 340 canalso function as a scalable codec.

The image and audio output unit 350 includes a display unit 351 and anaudio output unit 352.

The display unit 351 is a display unit that displays each image (forexample, the images 11 and 12 illustrated in FIG. 1) based on the imagedata decompressed by the image and audio decompression unit 340. As thedisplay unit 351, for example, a display panel such as an organic ELpanel, a crystal LED display, an LCD panel can be used. As the displayunit 351, a touch panel capable of performing a manipulation input whena user touches or approaches a display surface with his or her fingermay be used.

The audio output unit 352 is an audio output unit (for example, aspeaker) that outputs various audios (an audio and the like related toan image displayed on the display unit 351) based on the audio datadecompressed by the image and audio decompression unit 340. Here, as anaudio output method, for example, a method of reproducing only an audioof a source device allocated to a middle channel (a main image) from aspeaker and reproducing no audio of a source device allocated to aperipheral channel (a sub-image) can be used. As another audio outputmethod, for example, a method of setting the volume of an audio of asource device allocated to the middle channel as a main and lowering thevolume of an audio of a source device allocated to the peripheralchannel and reproducing the audio can be used. Other audio outputmethods may be used.

The user information acquisition unit 360 acquires information regardinga user (user information) and outputs the acquired user information tothe control unit 370. For example, the user information acquisition unit360 can acquire the user information by receiving an input from amanipulation reception unit (a keyboard, a mouse, a remote controller, agame pad, or a touch panel) for which the user can directly set adisplay method. The manipulation reception unit is, for example, amanipulation member that designates any region in an image displayed onthe display unit 351. For example, the user information acquisition unit360 can acquire the user information by receiving an input from a devicewhich can comprehend a user's intention, such as a camera, a microphone,or any of various sensors (for example, gyro sensors and sensorsdetecting human bodies).

For example, the user information acquisition unit 360 acquires the userinformation generated through a user motion when information based onthe stream received from another information processing device (forexample, the information processing device 200) using the wirelesscommunication is output from the image and audio output unit 350. Theuser information is, for example, user information generated through auser motion related to an image displayed on the display unit 351. Forexample, the user information is information generated based on a usermanipulation related to the image displayed on the display unit 351.

For example, the user information acquisition unit 360 can acquire theimage data generated by the imaging unit 361 (illustrated in FIG. 1) andgenerate user information. For example, the user information acquisitionunit 360 may acquire information (for example, position information andidentification information) acquired by an external device (for example,each sensor or a wearable device) and generate the user information.

The control unit 370 causes the management information retention unit390 to retain each piece of information acquired by the stream receptionunit 330 and manages each source device based on management informationretained the management information retention unit 390. The control unit370 performs the stream transmission control so that stability isimproved for streams transmitted from a plurality of source devices inthe entire system.

For example, the control unit 370 performs the stream transmissioncontrol based on the user information acquired by the user informationacquisition unit 360 and the management information retained in themanagement information retention unit 390. Specifically, the controlunit 370 generates a control signal for each source device to performthe stream transmission control based on the management informationretained in the management information retention unit 390 and outputsthe generated control signal to the control signal transmission unit380. For example, the control unit 370 changes the resolution of animage displayed on the display unit 351 based on the user informationand the management information and generates a control signal to requesta transmission rate equivalent to the resolution to each source device.For example, the control unit 370 generates a control signal to decide adisplay region of an image on the display unit 351 based on the userinformation and the management information. For example, the controlunit 370 generates a control signal to decide the size of an image onthe display unit 351 based on the user information and the managementinformation.

The control unit 370 performs control such that a frequency channel anda resolution to be used are set based on the user information and themanagement information. For example, the control unit 370 sets afrequency channel to be used for each source device in the plurality offrequency channels of the wireless communication unit 320. When thepower consumption mode is different from each frequency channel, thecontrol unit 370 comprehends each mode and sets the frequency channelfor caring the power consumption of a mobile device. That is, thecontrol unit 370 can separately set a first power consumption moderelated to the first frequency band and a second power consumption moderelated to the second frequency band of a higher data transmission speedthan the first frequency band.

The control signal transmission unit 380 performs a transmission processof transmitting the control signal output from the control unit 370 toanother wireless communication device via the wireless communicationunit 320 and the antenna 310.

The management information retention unit 390 is a table that retainsinformation (management information) to manage each source deviceconnected to the information processing device 300 using the wirelesscommunication. Content retained in the management information retentionunit 390 will be described in detail with reference to FIG. 4.

[Content Example Retained in Management Information Retention Unit]

FIG. 4 is a diagram schematically illustrating a content exampleretained in the management information retention unit 390 according tothe first embodiment of the present technology.

The management information retention unit 390 is a table that retainsinformation (management information) to manage each source deviceconnected to the information processing device 300 using the wirelesscommunication. For example, in the management information retention unit390, terminal identification information 391, a frequency channel 392, aradio wave propagation measurement information 393, device information394, a band use level 395, a display form 396, standby or wake-up 397,and multi-reception diversity correspondence 398 are retained inassociation therewith.

In the terminal identification information 391, identificationinformation is stored to identify the source devices connected to theinformation processing device 300 using the wireless communication.

In the frequency channel 392, a frequency channel actually used by thesource device connected to the information processing device 300 usingthe wireless communication is stored.

In the radio wave propagation measurement information 393, radio wavepropagation measurement information regarding the source deviceconnected to the information processing device 300 using the wirelesscommunication is stored. The radio wave propagation measurementinformation is measured by the stream reception unit 330 for each sourcedevice connected to the information processing device 300 using thewireless communication.

As the radio wave propagation measurement info illation 393, forexample, a packet error rate (PER), a bit error rate (BER), the numberof retransmissions of packets, and a throughput are stored. As the radiowave propagation measurement information 393, for example, frame drop, asignal to interference ratio (SIR), and a received signal strengthindicator (RSSI) are stored. Here, instead of the SIR, a signal tointerference plus noise ratio (SINR) may be used. The radio wavepropagation measurement information 393 illustrated in FIG. 4 is anexample. At least one piece of information among the pieces ofinformation may be stored or another piece of radio wave propagationmeasurement information may be measured by the stream reception unit 330to be stored. The radio wave propagation measurement informationmeasured by the source device may be acquired and stored. Packet delayreceived by a reception side may be determined and information regardingthe packet delay may be used as radio wave propagation measurementinformation. The packet delay serves as one index related to radio wavepropagation since delay occurs in transmission to the reception sidethrough a retransmission process in layer 2 at the time of occurrence ofan error. The packet delay serves as, for example, an index indicatingwhere link characteristics deteriorate in a wireless system in which aplurality of devices share wireless bands.

In the device information 394, classification of the source device (anattribute of the source device) connected to the information processingdevice 300 using the wireless communication is stored. For example,either of a mobile device and a stationary device is stored as theclassification of the source device. Either of a device of which a powersource is inserted and another device may be stored as theclassification of the source device. Either of a battery-driven deviceand another device may be stored as the classification of the sourcedevice.

In the band use level 395, a band use level of the source deviceconnected to the information processing device 300 using the wirelesscommunication is stored. As the band use level, for example, aresolution or a throughput can be used. For example, in the band uselevel, a throughput during use may be stored, a pre-decided table may beprepared, and a number indicating correspondence of a range of the tablemay be stored and managed.

In the display form 396, a data display form (an output form) based on astream transmitted from the source device connected to the informationprocessing device 300 using the wireless communication is stored. Forexample, a display form (a main image (a middle channel) or a sub-image(a peripheral channel)) of the image data which is displayed on thedisplay unit 351 and which is based on the stream transmitted from thesource device is stored. For example, an output form (a main audio or asub-audio) of the audio data which is output from the audio output unit352 and which is based on the stream transmitted from the source deviceis stored. A format in which the peripheral channel is not displayed maybe realized in accordance with the display form.

In the standby or wake-up 397, a mode (a standby mode or a wake-up mode)of the source device connected to the information processing device 300using the wireless communication is stored.

In the multi-reception diversity correspondence 398, informationindicating whether the source device connected to the informationprocessing device 300 using the wireless communication corresponds tothe multi-reception diversity is stored.

In this way, the management information retained in the managementinformation retention unit 390 is information for associating theidentification information (the terminal identification information 391)used to identify the other information processing device with thecapability information regarding the other information processing devicefor management. The management information includes at least theinformation (the radio wave propagation measurement information 393)regarding the radio wave propagation measurement related to thecommunication with the other information processing device and theinformation (the standby or wake-up 397) regarding power consumption asthe capability information regarding the other information processingdevice. The management information retained in the managementinformation retention unit 390 includes the information (the displayform 396) regarding a display form for displaying the image informationas the capability information regarding the other information processingdevice. The information regarding the display form is, for example,information indicating that the image information is displayed as maininformation or sub-information.

As described above, since the sink device side manages the informationrelated to the source device, it is possible to display the information(for example, the terminal identification information 391 or the name ofthe source device) related to the source device detectable by the sinkdevice. Further, it is possible to display the information (for example,the terminal identification information 391 or the name of the sourcedevice) related to the source device detectable by the sink device inassociation with the display screen of the source device. In otherwords, the control unit 370 can cause an image of the source device andinformation indicating the source device to be displayed on the displayunit 351 in association with each other. In this case, by causing theinformation (for example, the terminal identification information or thename) related to the source device to be displayed on (or nearby) a partof the image of the source device, it is possible to easily detect theuser and the display screen referred to by the user. Thus, the otheruser can recognize manipulatable source devices. In this case, forexample, the sink device can perform each control in the order in whichthe manipulation received.

[Transition Example of Image]

FIG. 5 is a diagram illustrating a transition example of imagesdisplayed on the display unit 351 of the information processing device300 according to the first embodiment of the present technology.

FIG. 5a illustrates an example of a display form in which the images 11and 12 are displayed on the display unit 351 of the informationprocessing device 300 by setting the image 11 as a middle channel andsetting the image 12 as a peripheral channel.

FIG. 5b illustrates an example of a display form in which the images 11and 12 are displayed on the display unit 351 of the informationprocessing device 300 by setting the image 11 as a peripheral channeland setting the image 12 as a middle channel.

For example, a case in which each of the information processing devices200 and 400 transmits a stream (the image data and the audio data) witha standard resolution to the information processing device 300 isassumed. In this case, as illustrated in FIG. 1, the image 11 based onthe image data from the information processing device 200 and the image12 based on the image data from the information processing device 400can be displayed on the display unit 351 of the information processingdevice 300 so that the sizes of the images 11 and 12 are the same. Inthis example, a given resolution and a display region are defined to bethe same, but a scaler function may be added to the display unit 351 sothat the images 11 and 12 are resealed and displayed on the display unit351. However, in the embodiments of the present technology, tofacilitate the description, this function is assumed not to be used inthe description.

In the display forms of the images 11 and 12, for example, the displayforms set at the time of the previous communication may be retained andthe images 11 and 12 may be displayed on the display unit 351 of theinformation processing device 300 according to the display forms.

The display forms of the images 11 and 12 may be decided based on anorder of connection to the information processing device 300. Forexample, a case in which the information processing device 200 is firstconnected to the information processing device 300 and the informationprocessing device 400 is connected to the information processing device300 after the connection is assumed. In this case, the images 11 and 12are displayed on the display unit 351 of the information processingdevice 300 by setting the image 11 as the middle channel and setting theimage 12 as the peripheral channel. That is, the images may be displayedin the procedure of the middle channel and the peripheral channel basedon the order of the connection to the information processing device 300.

As illustrated in FIG. 5a , when the images 11 and 12 are displayed onthe display unit 351 by setting the image 11 as the middle channel andsetting the image 12 as the peripheral channel, user information forsetting the image 12 as the middle channel is assumed to be acquired bythe user information acquisition unit 360. For example, when a viewerperforms a manipulation of setting the image 12 as the middle channelusing a pointer such as a remote controller or a gesture, the userinformation for setting the image 12 as the middle channel is acquiredby the user information acquisition unit 360. In this case, asillustrated in FIG. 5b , the images 11 and 12 are displayed on thedisplay unit 351 by setting the image 12 as the middle channel andsetting the image 11 as the peripheral channel. Further, displaypositions of the images 11 and 12 on the display surface of the displayunit 351 are decided based on user information (for example, a manualmanipulation or a line of sight) acquired by the user informationacquisition unit 360.

[Communication Example]

FIG. 6 is a sequence chart illustrating a communication process examplebetween devices included in the communication system 100 according tothe first embodiment of the present technology. FIG. 6 illustrates anexample of a communication process when a request for the displayposition of the information processing device 300 (the sink device) isperformed between the information processing device 200 (the sourcedevice) and the information processing device 300 (the sink device)through Capability Negotiation.

For example, when each request is performed at the time of CapabilityNegotiation from the information processing device 200 to theinformation processing device 300, the information processing device 300transmits a capability related to its own information processing device.For example, the information processing device 200 transmits aconfirmation request to the information processing device 300 (501 and502). The confirmation request is used to transmit a request of adisplay area size of the information processing device 200, an a displayarea size (however, it is not a transmission resolution) correspondingto multiple sources, and the display position (for example, coordinates)of the information processing device 200, and information related torotation of a display image and a request of confirmation related to aresolution/audio quality/power consumption performance. The confirmationrequest is an example, and other content may be requested at the sametime. Each piece of information may be transmitted at a time or may besequentially transmitted.

When the confirmation request is received (502), the informationprocessing device 300 transmits a confirmation response to theinformation processing device 200 (503 and 504). The confirmationresponse is used to transmit a response of the display area size of theinformation processing device 200, the display area size (however, it isnot a transmission resolution) corresponding to the multiple sources,and the display position of the information processing device 200 and aresponse of the rotation information of the display screen and theconfirmation related to the resolution/audio quality/power consumptionperformance. The confirmation response is an example, and other contentmay be transmitted as a response at the same time.

When the confirmation response is received (504), the informationprocessing device 200 transmits a mode request (including a designationof the display position) to the information processing device 300 (505and 506). Thus, the information processing device 200 can designate thedisplay position for the information processing device 300.

When the mode request is received (506), the information processingdevice 300 transmits a mode setting completion notification (including adesignation of the display position) to the information processingdevice 200 (507 and 508). Thus, the information processing device 200can cause an image to be displayed at the designated display position.

When the mode setting completion notification is received (508), theinformation processing device 200 transmits an image (image data)serving as a display target to the information processing device 300(509, 510). In this case, the information processing device 200transmits the image (image data) serving as the display target to theinformation processing device 300 based on the received mode settingcompletion notification. Then, the information processing device 300causes the received image to be displayed on the display unit 351 (511).

As described above, in the interchange of the capability between thesource device and the sink device, the sink device can request thedisplay area size, the display area size (however, it is not atransmission resolution) corresponding to the multiple sources, thedisplay position, and the rotation information of the display screen.Thus, it is possible to notify the source device of the display positionof the sink device in real time, and it is possible to control thedisplay position from the source device. In other words, even in themulti-source environment, it is possible to arrange an appropriatedisplay window (an image display region of the source device) at anappropriate position.

[Communication Example]

FIGS. 7 and 8 are a sequence chart illustrating a communication processexample between devices included in the communication system 100according to the first embodiment of the present technology. FIGS. 7 and8 illustrate a communication example when the interchange illustrated inFIG. 6 is performed in the topology illustrated in FIG. 1.

In FIGS. 7 and 8, the image and audio signal generation unit 250, theimage and audio compression unit 260, and the stream transmission unit270 among the units included in the information processing device 200are illustrated as a data transmission system 201. The antenna 210, thewireless communication unit 220, the control signal reception unit 230,and the control unit 240 are illustrated as a line control system 202.Even in the information processing device 400, similarly, they areillustrated as a data transmission system 401 and a line control system402.

In FIGS. 7 and 8, the antenna 310, the wireless communication unit 320,the stream reception unit 330, the control unit 370, and the controlsignal transmission unit 380 in the configuration of the informationprocessing device 300 are illustrated as a line control system 301. Theimage and audio decompression unit 340, the image and audio output unit350, and the user information acquisition unit 360 are illustrated as aninput and output system 302.

First, when the information processing device 300 is powered up, aprevious display form (which is a display form when the informationprocessing device 300 is powered down) is set as a display form (whichis an image display form and an audio display form) of the informationprocessing device 300 (1001). The control unit 370 of the informationprocessing device 300 causes the management information retention unit390 to retain the management information of each source device connectedto the information processing device 300 using the wirelesscommunication (illustrated in FIG. 4). As illustrated in FIG. 5, thecontrol unit 370 of the information processing device 300 causes thedisplay unit 351 to display the images 11 and 12 corresponding to twostreams respectively transmitted from the information processing devices200 and 400 based on the previous display form.

Subsequently, a case in which the user performs a manipulation ofsetting the display form (changing manipulation) is assumed (1002). Inthis case, a control signal related to the setting manipulation isacquired as the user information by the user information acquisitionunit 360 and the user information is output to the control unit 370.Then, the control unit 370 changes the content retained in themanagement information retention unit 390 (illustrated in FIG. 4) basedon the user information (1003 and 1004). For example, as illustrated inFIG. 5b , a case in which the setting manipulation (changingmanipulation) is performed to set the image 11 based on the image datafrom the information processing device 200 as the peripheral channel isassumed. In this case, the control unit 370 changes the display form 396(illustrated in FIG. 4) of the information processing device 200 in themanagement information retention unit 390 to “SUB” (1003 and 1004).

At the time of the Capability Negotiation from the informationprocessing device 200 to the information processing device 300, eachrequest can be transmitted, and the information processing device 300can transmit the capability related to its own information processingdevice. For example, the information processing device 200 transmits afirst confirmation request to the information processing device 300(1005 and 1006). The first confirmation request is used to request thedisplay area size of the information processing device 200, the displayarea size (however, it is not a transmission resolution) correspondingto the multiple sources, the display position of the informationprocessing device 200, and the information related to the rotation ofthe display image.

When the first confirmation request is received (1006), the informationprocessing device 300 transmits a first confirmation response to theinformation processing device 200 (1007 to 1010). The first confirmationresponse is used to transmit a response of the display area size of theinformation processing device 200, the display area size (however, it isnot a transmission resolution) corresponding to the multiple sources,the display position of the information processing device 200, and theinformation related to the rotation of the display image.

When the first confirmation response is received (1008), the informationprocessing device 200 transmits the mode request (including adesignation of the display position) to the information processingdevice 300 (1011 and 1012). Thus, the information processing device 200can designate the display position for the information processing device300.

When the mode request is received (1012), the information processingdevice 300 transmits the mode setting completion notification (includinga designation of the display position) to the information processingdevice 200 (1013 to 1016). Thus, the information processing device 200can cause the image to be displayed at the designated display position.

The information processing device 300 may be powered on, and theinterchange between the information processing device 300 and theinformation processing device 400 may be performed. In this case,similarly to the process (1001 to 1016), the process (1017 to 1032) isperformed between the information processing device 300 and theinformation processing device 400.

The first confirmation request and the first confirmation response arean example, and the first confirmation request and the firstconfirmation response may be interchanged at the same time or may beinterchanged in different orders. Further, other content may besimultaneously or sequentially transmitted.

As illustrated in FIGS. 7 and 8, there is a possibility that a pluralityof source devices (the information processing device 200 and theinformation processing device 400) will transmit the mode request (themode request illustrated in FIGS. 7 (1011 and 1012) and the mode requestillustrated in FIGS. 8 (1027 and 1028)). In this case, there is apossibility that a plurality of source devices will transmit the moderequest at the same time. In this regard, a control example of the sinkdevice when the mode requests are received from a plurality of sourcedevices at the same time (or substantially at the same time).

For example, when the mode requests are received from the sourcedevices, the sink device (for example, the information processing device300) can decide the mode request of the source device that ispreferentially processed according to display content of the displayunit. In this case, the following two types of decision methods (R1) and(R2) are considered:

(R1) a method in which the sink device receives the mode requests fromthe source devices in parallel and perform selection based on thereceived information; and

(R2) a method in which the sink device bundles the mode requeststransmitted from the source devices and manages a bundle of moderequests as one piece of information using an ID.

In the method (R1), the sink device can directly detect the moderequests transmitted from the source devices. Thus, the sink device cancontrol such that the mode requests transmitted from the source devicesare turned on or off. Further, the sink device can decide to give apriority to content of the mode request transmitted from a certainsource device. However, in the method (R1), as the number of sourcedevices increases, a processing load of the sink device increases.

In the method (R2), the sink device bundles the mode requeststransmitted from a plurality of source devices and then switches contentof the mode requests. For this reason, the processing load of the sinkdevice can be reduced. However, in the method (R2), the mode requeststransmitted from the source devices are bundled as managed using an ID.Thus, in order for the sink device to perform control such that the moderequests transmitted from the source devices are turned on or off, aheader identifying each source device is necessary.

Here, in the image transmission standard, it is often decided by a moderequest content switching unit (illustrated in FIGS. 9 to 14) thatswitches content of the mode request according to the standard. To thisend, for example, ports for a transmission control protocol (TCP), auser datagram protocol (UDP), and real time streaming protocol (RTSP),and WebSocket are considered to be prepared and managed.

[On-Off Control Example of Forward Channel]

FIG. 9 is a diagram illustrating a configuration example when the sinkdevice performs on-off control of the mode request according to thefirst embodiment of the present technology.

FIG. 9 illustrates an example in which on-off control of the moderequest is performed between one sink device 1100 and a plurality ofsource devices 1211 to 1214 according to the decision method (R1). Inother words, an example in which the sink device 1100 prepares aplurality of ports for a plurality of source devices 1211 to 1214, andselects the source device based on a request from an upper layer and astate of the display screen. For example, the sink device 1100 may setan Off operation of the source device based on an Off request from theupper layer. Further, for example, the sink device 1100 may set the Offoperation of the source device based on a display state of the displayscreen in the sink device 1100.

Each of the source devices 1211 to 1214 transmits forward channelinformation (for example, the mode request) to the sink device 1100(1215 to 1218).

The sink device 1100 includes a display control unit 1110 and a moderequest content switching unit 1120. The display control unit 1110 andthe mode request content switching unit 1120 correspond to the streamreception unit 330 and the control unit 370 illustrated in FIG. 3.

The display control unit 1110 decides an input from a source device towhich switching is performed based on the display position of the image(the image transmitted from the source device) to be displayed on thedisplay unit. The display control unit 1110 notifies the mode requestcontent switching unit 1120 of the decided content (1111).

The display control unit 1110 decides an input from a source device thatis turned on and an input from a source device that is turned off basedon the display position. After the sink device 1100 and the sourcedevice are linked, a stop operation, a reproduction operation, adisconnection operation, a start operation, and the like can beappropriately performed through the interchange between the sink device1100 and the source device.

The display control unit 1110 can implement exclusive control byignoring the forward channel information for the ports of the sourcedevices 1211 to 1214. The display control unit 1110 can implementexclusive control by outputting a transmission stop message to theports.

The mode request content switching unit 1120 switches and receives theforward channel information transmitted from the source devices based onthe content decided by the display control unit 1110, and outputs thereceived information to the display control unit 1110. For example, themode request content switching unit 1120 receives only the forwardchannel information transmitted from the source device decided by thedisplay control unit 1110, and outputs the information to the displaycontrol unit 1110.

FIG. 10 is a diagram illustrating a configuration example when the sinkdevice performs on-off control of the mode request according to thefirst embodiment of the present technology.

FIG. 10 illustrates an example in which the on-off control of the moderequest is performed between one sink device 1300 and a plurality ofsource devices 1211 to 1214 according to the decision method (R2). Inother words, the sink device 1300 prepares one port for a bundle offorward channel information, and manages the source devices based on anID header inserted into a packet. The source devices 1211 to 1214 arethe same as the source devices 1211 to 1214 illustrated in FIG. 9.

The sink device 1300 includes a display control unit 1310, a moderequest content switching unit 1320, and a common input switching unit1330. The display control unit 1310, the mode request content switchingunit 1320, and the common input switching unit 1330 correspond to thestream reception unit 330 and the control unit 370 illustrated in FIG.3.

The display control unit 1310 decides an input from a source devicewhich switching is performed based on the display position of the image(the image transmitted from the source device) to be displayed on thedisplay unit. The display control unit 1310 notifies the mode requestcontent switching unit 1320 of the decided content (1311).

The display control unit 1310 decides an input from a source device thatis turned on and an input from a source device that is turned off basedon the display position. After the sink device 1300 and the sourcedevice are linked, a stop operation, a reproduction operation, adisconnection operation, a start operation, and the like can beappropriately performed through the interchange between the sink device1300 and the source device.

The mode request content switching unit 1320 receives the forwardchannel information transmitted from the source devices switched by thecommon input switching unit 1330 based on the content decided by thedisplay control unit 1310. Then, the mode request content switching unit1320 output the received information to the display control unit 1310.

The common input switching unit 1330 receives information correspondingto a predetermined standard and outputs the received information to themode request content switching unit 1320. For example, the common inputswitching unit 1330 receives, for example, the ID information of eachsource device and control data.

The common input switching unit 1330 performs switching as to an inputfrom a source device that is turned on and an input from a source devicethat is turned off based on the ID information of each source devicethat is decided in advance and the request transmitted from the moderequest content switching unit 1320.

[On-Off Control Example of UIBC]

The decision methods can be applied to the UIBC. In this regard, anon-off control example of the UIBC will be described below.

FIG. 11 is a diagram illustrating a configuration example when the sinkdevice performs the on-off control of the UIBC according to the firstembodiment of the present technology.

FIG. 11 illustrates an example in which the on-off control of the UIBCis performed between one sink device 1100 and a plurality of sourcedevices 1211 to 1214 according to the decision method (R1). In otherwords, an example in which the sink device 1100 prepare a plurality ofports for a plurality of source devices 1211 to 1214, and selects theUIBC based on the request from the upper layer or the state of thedisplay screen. For example, the sink device 1100 can set an off stateof the UIBC based on the Off request from the upper layer. Further, forexample, the sink device 1100 can set the off state of the UIBC based onthe display state of the display screen in the sink device 1100.

Each of the source devices 1211 to 1214 transmits UIBC information tothe sink device 1100. The sink device 1100 transmits the UIBCinformation to each of the source devices 1211 to 1214 (1225 to 1228).

The display control unit 1110 decides an input from a source devicewhich switching is performed based on the display position of the image(the image transmitted from the source device) to be displayed on thedisplay unit. The display control unit 1110 notifies the mode requestcontent switching unit 1120 of the decided content (1112).

The display control unit 1110 decides an input from a source device thatis turned on and an input from a source device that is turned off basedon the display position. After the sink device 1100 and the sourcedevice are linked, a stop operation, a reproduction operation, adisconnection operation, a start operation, and the like can beappropriately performed through the interchange between the sourcedevice 1100 and the sink device.

The display control unit 1110 can implement exclusive control byignoring the UIBC information for the ports of the source devices 1211to 1214. The display control unit 1110 can implement exclusive controlby outputting a transmission stop message to the ports.

The mode request content switching unit 1120 switches and receives theUIBC information transmitted from the source devices based on thecontent decided by the display control unit 1110, and outputs thereceived information to the display control unit 1110. For example, themode request content switching unit 1120 receives only the UIBCinformation transmitted from the source device decided by the displaycontrol unit 1110, and outputs the information to the display controlunit 1110.

FIG. 12 is a diagram illustrating a configuration example when the sinkdevice performs the on-off control of the UIBC according to the firstembodiment of the present technology

FIG. 12 illustrates an example in which the on-off control of the UIBCis performed between one sink device 1300 and a plurality of sourcedevices 1211 to 1214 according to the decision method (R2). In otherwords, the sink device 1300 prepares one port for a bundle of UIBCinformation, and manages the source devices based on an ID headerinserted into a packet.

The display control unit 1310 decides an input from a source device towhich switching is performed based on the display position of the image(the image transmitted from the source device) to be displayed on thedisplay unit. The display control unit 1310 notifies the mode requestcontent switching unit 1320 of the decided content (1312).

The display control unit 1310 decides an input from a source device thatis turned on and an input from a source device that is turned off basedon the display position. After the sink device 1300 and the sourcedevice are linked, a stop operation, a reproduction operation, adisconnection operation, a start operation, and the like can beappropriately performed through the interchange between the sourcedevice 1300 and the sink device.

The mode request content switching unit 1320 receives the UIBCinformation transmitted from the source devices switched by the commoninput switching unit 1330 based on the content decided by the displaycontrol unit 1310. Then, the mode request content switching unit 1320output the received information to the display control unit 1310.

The common input switching unit 1330 receives information correspondingto a predetermined standard and outputs the received information to themode request content switching unit 1320. For example, the common inputswitching unit 1330 receives, for example, the ID information of eachsource device and control data. The common input switching unit 1330decides an input from a source device that is turned on and an inputfrom a source device that is turned off based on the ID information ofeach source device that is decided in advance and the requesttransmitted from the mode request content switching unit 1320. Forexample, the mode request content switching unit 1320 receives only theUIBC information transmitted from the source device decided by thedisplay control unit 1310, and outputs the information to the displaycontrol unit 1310.

[On-Off Control Example of UIBC at Time of Multi-Sink]

Next, the on-off control example of the UIBC when a plurality of sourcedevices are connected with the sink device will be described.

FIG. 13 is a diagram illustrating a configuration example when the sinkdevice performs the on-off control of the UIBC according to the firstembodiment of the present technology.

FIG. 13 illustrates an example in which the on-off control of the UIBCis performed between a plurality of sink devices 1501 to 1504 and onesource device 1400 according to the decision method (R1). In otherwords, an example in which the source device 1400 prepares a pluralityof ports for a plurality of sink devices 1501 to 1504, and the UIBC isselected based on the request from the upper layer or the manipulationof the user on the display screen is illustrated. For example, thesource device 1400 can perform the Off request for the UIBC based on theOff request from the upper layer. Further, for example, the sourcedevice 1400 can perform the Off request for the UIBC based on themanipulation of the user on the display screen in the sink device 1100.

Each of the sink devices 1501 to 1504 transmits the UIBC information tothe source device 1400 (1505 to 1508).

A display control unit 1410 decides an input from a sink device to whichswitching is performed based on the display position of the image (theimage transmitted from the source device) to be displayed on the displayunit of each of the sink devices 1501 to 1504. The display control unit1410 notifies a mode request content switching unit 1420 of the decidedcontent (1411).

The display control unit 1410 decides an input from a sink device thatis turned on and an input from a sink device that is turned off based onthe display position. After the source device 1400 and the sink deviceare linked, a stop operation, a reproduction operation, a disconnectionoperation, a start operation, and the like can be appropriatelyperformed through the interchange between the source device 1400 and thesink device.

The display control unit 1410 can implement exclusive control byignoring the UIBC information for the ports of the sink devices 1501 to1504. The display control unit 1410 can implement exclusive control byoutputting a transmission stop message to the ports.

The mode request content switching unit 1420 switches and receives theUIBC information transmitted from the sink devices based on the contentdecided by the display control unit 1410, and outputs the receivedinformation to the display control unit 1410. For example, the moderequest content switching unit 1420 receives only the UIBC informationtransmitted from the sink device decided by the display control unit1410, and outputs the information to the display control unit 1410.

FIG. 14 is a diagram illustrating a configuration example when the sinkdevice performs the on-off control of the UIBC according to the firstembodiment of the present technology.

FIG. 14 illustrates an example in which the on-off control of the UIBCis performed between a plurality of sink devices 1501 to 504 and onesource device 1600 according to the decision method (R2). In otherwords, the source device 1600 prepares one port for a bundle of UIBCinformation, and manages the sink devices based on an ID header insertedinto a packet.

A display control unit 1610 decides an input from a sink device to whichswitching is performed based on the display position of the image (theimage transmitted from the source device) displayed on the display unitof each of the sink device. The display control unit 1610 notifies amode request content switching unit 1620 of the decided content (1611).

The display control unit 1610 decides an input from a sink device thatis turned on and an input from a sink device that is turned off based onthe display position. After the source device 1600 and the source deviceare linked, a stop operation, a reproduction operation, a disconnectionoperation, a start operation, and the like can be appropriatelyperformed through the interchange between the source device 1600 and thesink device.

The mode request content switching unit 1620 receives the UIBCinformation transmitted from the sink devices switched by a common inputswitching unit 1630 based on the content decided by the display controlunit 1610. Then, the mode request content switching unit 1620 outputsthe received information to the display control unit 1610 (1611).

The common input switching unit 1630 receives information correspondingto a predetermined standard, and outputs the received information to themode request content switching unit 1620 (1621). For example, the commoninput switching unit 1630 receives, for example, the ID information ofeach sink device and the control data. The common input switching unit1630 decides an input from a sink device that is turned on and an inputfrom a sink device that is turned off based on the ID information ofeach sink device that is decided in advance and the request transmittedfrom the mode request content switching unit 1620. For example, the moderequest content switching unit 1620 receives only the UIBC informationtransmitted from the sink device decided by the display control unit1610, and outputs the information to the display control unit 1610.

As described above, the control data and the UIBC information can beinterchanged between the sink device and the source device. Further, forexample, since the control data and the UIBC information can beefficiently interchanged between the sink device and the source device,it is possible to share the port or use the WebSocket, for example. Inthis regard, an example in which each piece of information isinterchanged between the source device and the sink device using theWebSocket will be described with reference to FIGS. 20 and 21.

The present technology is not limited to this example. For example, inthe embodiment of the present technology, the on-off control examples ofthe forward channel, the control data, and the UIBC information havebeen described, but the present technology is not limited thereto. Forexample, the same on-off control may be performed even in the RTSPcommand (a play/pause, a standby/resume, an IDR refresh request, or thelike) illustrated in FIG. 18 and FIG. 19. A collaboration operation canbe also performed between devices such that the on-off control isperformed during a short period so that the on-off control is notrecognized from a point of view of the user. The on-off control mayrefer to a technique of controlling a connection, a disconnection, astart, a stop, validation (reception), and invalidation (ignoring).

For example, a device (for example, a display position control client)other than the source device and the sink device may be added, and thedevice may manipulate the source devices through the sink device. Thisexample is illustrated in FIGS. 15 and 16. In other words, FIGS. 15 and16 illustrates an example in which an information processing device 203(a display position control client) is added, and the informationprocessing device 203 manipulates the information processing device 200(the source device) through the information processing device 300 (thesink device).

Here, the added device (the device other than the source device and thesink device) may be one or more input device such as a keyboard, amouse, a joystick or may be connected with the sink device in a wired orwireless manner. A plurality of input devices may be fixedly orremovably installed in the sink device. As described above, when thesink device includes a plurality of input devices (for example,including a removable, wired, or wireless connection) manipulationinformation of a plurality of input devices is included in the UIBCinformation and transmitted through one or more sink devices. At thistime, ID information inserted into a packet of the UIBC information maybe inserted as a plurality of pieces of ID information related to aplurality of input devices.

[Communication Example]

FIGS. 15 and 16 are sequence charts illustrating a communication processexample between the devices included in the communication system 100according to the first embodiment of the present technology.

FIGS. 15 and 16 illustrate an example of a communication process whenthe display position control client (the information processing device(the source device) 203) other than the information processing device200 (the source device) and the information processing device 300 (thesink device) detects the display position of the sink device. Thedisplay position control client (the information processing device 203)is a control client that detects the display position of the informationprocessing device 300 (the sink device), and causes a display positionimage of the information processing device 300 (the sink device) to bedisplayed on a display device (not illustrated). For example, thedisplay position and the size of each image displayed on the informationprocessing device 300 (the sink device are displayed on the displaydevice for each source device. Thus, the user of the source device canrecognize each image.

In FIGS. 15 and 16, the image and audio signal generation unit 250, theimage and audio compression unit 260, and the stream transmission unit270 among the units included in the information processing device 200are illustrated as a data transmission system 201. The antenna 210, thewireless communication unit 220, the control signal reception unit 230,and the control unit 240 are illustrated as a line control system 202.

In FIGS. 15 and 16, the antenna 310, the wireless communication unit320, the stream reception unit 330, the control unit 370, and thecontrol signal transmission unit 380 in the configuration of theinformation processing device 300 are illustrated as a line controlsystem 301. The image and audio decompression unit 340, the image andaudio output unit 350, and the user information acquisition unit 360 areillustrated as an input and output system 302.

First, when the information processing device 300 is powered up, aprevious display form (which is a display form when the informationprocessing device 300 is powered down) is set as a display form (whichis an image display form and an audio display form) of the informationprocessing device 300 (521). The control unit 370 of the informationprocessing device 300 causes the management information retention unit390 to retain the management information of each source device connectedto the information processing device 300 using the wirelesscommunication (illustrated in FIG. 4). As illustrated in FIG. 5, thecontrol unit 370 of the information processing device 300 causes thedisplay unit 351 to display the images 11 and 12 corresponding to twostreams respectively transmitted from the information processing devices200 and 400 based on the previous display form.

Subsequently, a case in which the user performs a manipulation ofsetting the display form (changing manipulation) is assumed (522). Inthis case, a control signal related to the setting manipulation isacquired as the user information by the user information acquisitionunit 360 and the user information is output to the control unit 370.Then, the control unit 370 changes the content retained in themanagement information retention unit 390 (illustrated in FIG. 4) basedon the user information (523 and 524). For example, as illustrated inFIG. 5b , a case in which the setting manipulation (changingmanipulation) is performed to set the image 11 based on the image datafrom the information processing device 200 as the peripheral channel isassumed. In this case, the control unit 370 changes the display form 396(illustrated in FIG. 4) of the information processing device 200 in themanagement information retention unit 390 to “SUB” (523 and 524).

Further, at the time of the Capability Negotiation from the informationprocessing device 200 to the information processing device 300, eachrequest can be performed, and the information processing device 300 cantransmit the capability related to its own information processingdevice. For example, the information processing device 200 transmits thefirst confirmation request to the information processing device 300 (525and 526). The first confirmation request is used to request the displayarea size of the information processing device 200, the display areasize (however, it is not a transmission resolution) corresponding to themultiple sources, and the display position of the information processingdevice 200.

When the first confirmation request is received (526), the informationprocessing device 300 transmits the first confirmation response to theinformation processing device 200 (527 to 530). The first confirmationresponse is used to transmits a response of the display area size of theinformation processing device 200, the display area size (however, it isnot a transmission resolution) corresponding to the multiple sources,and the display position of the information processing device 200. Theinformation processing device 300 transmits the first confirmationresponse to the information processing device 203 (531 and 532). In thiscase, the information processing device 300 may include each piece ofinformation (the display area size of another source device, the displayarea size corresponding to the multiple sources, and the displayposition of another source device) of another source device other thanthe information processing device 200 in the first confirmation responseand transmit the resulting first confirmation response to theinformation processing device 203. Thus, the information processingdevice 203 can rapidly detect the display position of each source devicein the sink device.

The information processing device 200 transmits the mode request(including a designation of the display position) to the informationprocessing device 300 (533, 534). Thus, the information processingdevice 200 can designate the display position for the informationprocessing device 300.

When the mode request is received (534), the information processingdevice 300 transmits the mode setting completion notification (includinga designation of the display position) to the information processingdevice 200 (535 to 538). The information processing device 300 transmitsthe mode setting completion notification (including a designation of thedisplay position) to the information processing device 203 (539 and540). Thus, the information processing device 203 can cause the image tobe displayed at the designated display position.

The information processing device 200 transmits a second confirmationrequest to the information processing device 300 (541 and 542). Thesecond confirmation request is used to request the confirmation relatedto the resolution/audio quality/power consumption performance, and forexample, the first confirmation request may be an operation of a futurestandard of the Wi-Fi standard (for example, Wi-Fi CERTIFIED Miracast),and the second confirmation request may be an operation within thecurrent Wi-Fi standard (for example, Wi-Fi CERTIFIED Miracast).

When the second confirmation request is received (542), the informationprocessing device 300 transmits a second confirmation response to theinformation processing device 200 (543 and 544). The second confirmationresponse is used to transmit a response of the confirmation related tothe resolution/audio quality/power consumption performance.

The first and second confirmation requests and the first and secondconfirmation responses are an example, and the first and secondconfirmation requests and the first and second confirmation responsesmay be interchanged at the same time or may be interchanged in differentorder. Further, other content may be simultaneously or sequentiallytransmitted.

When the second confirmation response is received (544), the linecontrol system 202 of the information processing device 200 outputs amode setting (including a designation of the display position) to thedata transmission system 201 (545 and 546). The line control system 202of the information processing device 200 transmits the mode setting(including a designation of the display position) to the informationprocessing device 300 (547 and 548). The mode setting is used to performa setting related to resolution/audio quality/power consumptionperformance.

Here, the display position of the image transmitted from the informationprocessing device 200 is assumed to be changed from the middle screen(the main screen) to the peripheral screen (the sub screen) (549). Inthis case, in the information processing device 200, settings forchanging to the peripheral screen (the sub screen) are performed (550and 551). The information processing device 203 displays the image (theimage of the information processing device 200) of the display positionin the information processing device 300 as the sub screen. Theinformation processing device 203 transmits a command (a displayposition manipulation command) for manipulating the display position ofthe information processing device 200 to the information processingdevice 300 (552 and 553).

The information processing device 300 receives a stream from theinformation processing device 200 (554 and 555), and causes an imagebased on the stream to be displayed on the display unit 351 (556). Then,when the change to the display position requested by the displayposition manipulation command is performed, the information processingdevice 300 performs a setting change so that a resolution, an audio, andpower consumption that are optimum for the information processing device300 are set. Further, when the setting manipulation (the changemanipulation) of the display form by the user is performed (557), eachprocess is performed according to the setting manipulation (558 and559). Then, the information processing device 300 transmits a settingchange request for performing the setting change to the informationprocessing devices 200 and 203 (560 to 563).

The information processing device 300 receives a stream from theinformation processing device 200 (564 and 565).

As described above, it is possible to notify the source device of thedisplay position of the sink device in real time, and it is possible tocontrol the display position from the source device.

In the example illustrated in FIGS. 15 and 16, a plurality of sourcedevices are connected to the sink device, but the embodiment of thepresent technology is not limited to this example. For example, it ispossible to correspond even to a connection environment in which onlyone source device is connected to the sink device (the informationprocessing device 300).

As the screen displayed by the sink device (the information processingdevice 300), the sink device (the information processing device 300) mayreceive and display not only the image received from the source device(the information processing device 200) but also the image informationreceived from another connection line with which the sink device (theinformation processing device 300) is equipped. For example, it ispossible to correspond to even when an information processing devicethat conforms to the Hybridcast standard is used as the sink device. Forexample, a device (for example, a video viewing device) that conforms tothe Hybridcast standard can receive auxiliary information (including amoving image) from a communication line while receiving an image (video)from a broadcast wave. In this regard, the image of the source device(the information processing device 200) may be displayed in a part of anenvironment that conforms to the Hybridcast standard. This example willbe described in a fourth embodiment of the present technology.

Further, the information processing device 300 may perform control suchthat a sum data transmission speed of the two streams transmitted fromeach of the information processing devices 200 and 400 is minimized. Forexample, a maximum allowable value of the sum data transmission speed isset in the control unit 370 of the reception side information processingdevice 300. After the control unit 370 transmits a change request fordecreasing a bit rate to the information processing device 200, thecontrol unit 370 acquires bit rates of two streams transmitted from theinformation processing devices 200 and 400 from the stream receptionunit 330. Subsequently, the control unit 370 calculates a sum datatransmission speed of the acquired two streams. Subsequently, thecontrol unit 370 decides the bit rate of the streams transmitted fromthe information processing device 400 within a range not greater thanthe set maximum allowable value and transmits a change request forincreasing the bit rate to the information processing device 400. Whenthe packet error rate (PER) is large and thus is not receivable in thesame frequency channel despite of the setting of the minimum bit rate, adifferent frequency channel may be used. When the images (the middlechannel and the peripheral channel) are paused for a given time or more,the image data may be stopped as long as a manipulation (for example,pointing) is not performed by the user.

In this way, according to the first embodiment of the presenttechnology, even when one sink device receives a plurality of streamstransmitted from a plurality of source devices, appropriate streamtransmission control (for example, the data transmission speed control)can be performed according to a manipulation, a situation, and anintention of the user. For example, it is possible to decrease the datatransmission speeds of some of the plurality of image and audio streamsand increase the data transmission speeds of the remaining streamsaccording to an operation, a situation, and an intention of the user.

For example, when a sink device receives and displays a plurality ofstreams, important images and audios with high quality set timely by theuser can be enjoyed. For the otherwise images and audios, the datatransmission speeds can be adjusted automatically to the optimumfrequency channel, power consumption, and transmission rate. Further, itis possible to manipulate the connection control in real time whilestably operating the image transmission mode.

Here, for the management information retained in the managementinformation retention unit 390, a command prepared in Wi-Fi CertifiedMiracast can be used for exchanging the management information. In thiscase, capability negotiation or capability re-negotiation defined in theWi-Fi Display specification can be used. Here, as capability negotiationor capability re-negotiation, for example, RFC5939 or the Wi-FiCertified Miracast specification can be exemplified. However, capabilitynegotiation or capability re-negotiation is not limited thereto, but isdefined as interchange of the device performance information. Acommunication example of the interchange using a command of the Wi-FiCertified Miracast specification is illustrated in FIGS. 17 to 19.

[Communication Example of Interchange Using Wi-Fi Certified MiracastSpecification Command]

FIGS. 17 to 19 are sequence charts illustrating a communication processexample between a source device 410 and a sink device 420 according tothe first embodiment of the present technology. FIGS. 17 to 19illustrate a communication example of interchange using an RTSPprotocol. A source device 410 corresponds to the information processingdevices 200 and 400 and a sink device 420 corresponds to the informationprocessing device 300.

First, the description will be made with reference to FIG. 17. Forexample, as indicated by a dotted rectangle 430 of FIG. 17, an “RTSP M3Request” (RTSP GET_PARAMETER Request) message transmitted from thesource device 410 to the sink device 420 and an “RTSP M3 Response” (RTSPGET_PARAMETER Response) message transmitted from the sink device 420 tothe source device 410 in response to the “RTSP M3 Request” (RTSPGET_PARAMETER Request) message can be used.

On the other hand, the messages may be appropriately transmitted fromthe source device 410 to the sink device 420. For example, theinterchange of the “RTSP M3 Request” (RTSP GET_PARAMETER Request)message and the “RTSP M3 Response” (RTSP GET_PARAMETER Response) messagemay be omitted, the management information may be included in a messageto be transmitted from the source device 410 to the sink device 420, themanagement information may be transmitted from the source device 410 tothe sink device 420, and the sink device may select the information andretain the information in the management information retention unit 390.For example, when the content protection setting is performed, linkprotection setup is performed after M3 Response. Therefore, it isdesirable to perform communication while ensuring a secrecy ability of alink set once by transmitting only messages of M4 or higher.

The interchange of information regarding the power consumption mode canbe performed with predetermined messages using the RTSP protocol. Forexample, three kinds of management information of the following (1) to(3) can be exchanged.

(1) “setting to the standby mode,”

(2) “when the source device cancels the standby mode or when the sourcedevice cancels the standby mode of the sink device,” and

(3) “when the sink device cancel the standby mode or when the sinkdevice cancels the standby mode of the source device.”

Now, the description will be made with reference to FIG. 18. Forexample, when a command prepared in Wi-Fi Certified Miracast is used, an“RTSP M12 Request” (RTSP SET_PARAMETER (with WFD-standby)) messagetransmitted from the source device 410 to the sink device 420 and an“RTSP M12 Response” (RTSP OK) message transmitted from the sink device420 to the source device 410 in response to the “RTSP M12 Request” (RTSPSET_PARAMETER (with WFD-standby)) message can be used in the interchangeof (1) “setting to the standby mode,” as described above. On the otherhand, the same also applies to the setting to the standby mode from thesink device 420 to the source device 410.

Next, the description will be made with reference to FIG. 19. Forexample, (2) “when the source device cancels the standby mode or whenthe source device cancels the standby mode of the sink device,” asdescribed above, the source device 410 interchanges an “RTSP M5 Request”(RTSP SET_PARAMETER (Request (wfd-trigger-method: PLAY)) messagetransmitted to the sink device 420 and an “RTSP M5 Response” (RTSP OK)message transmitted from the sink device 420 to the source device 410 inresponse to the “RTSP M5 Request” (RTSP SET_PARAMETER (Request(wfd-trigger-method: PLAY)) message. The sink device 420 can uses an“RTSP M7 Request” (RTSP PLAY Request) message transmitted to the sourcedevice 410 and an “RTSP M7 Response” (RTSP OK) message transmitted fromthe source device 410 to the sink device 420 in response to the “RTSP M7Request” (RTSP PLAY Request) message.

For example, (3) “when the sink device cancel the standby mode or whenthe sink device cancels the standby mode of the source device,” asdescribed above, the sink device 420 can use the “RTSP M7 Request” (RTSPPLAY Request) message transmitted to the source device 410 and the “RTSPM7 Response” (RTSP OK) message transmitted from the source device 410 tothe sink device 420 in response to the “RTSP M7 Request” (RTSP PLAYRequest) message.

In this way, the wireless communication unit 320 can perform theexchange of the capability information with capability negotiation orcapability re-negotiation defined in the Wi-Fi Display specification.For example, the capability information is exchanged with the RTSP M3message in capability negotiation or capability re-negotiation.

In this way, for example, the wireless communication unit 320 of theinformation processing device 300 performs the communication with theinformation processing device 200 to exchange the capability informationregarding the information processing device 300 and the capabilityinformation regarding the information processing device 200. Thewireless communication unit 220 of the information processing device 200performs the communication with the information processing device 300 toexchange the capability information regarding the information processingdevice 200 and the capability information regarding the informationprocessing device 300. In this case, the wireless communication units220 and 320 can exchange the capability information with capabilitynegotiation or capability re-negotiation.

The control unit 370 of the information processing device 300 performsthe stream transmission control with the information processing device200 based on the capability information regarding the informationprocessing device 200, the radio wave propagation measurementinformation regarding the communication with the information processingdevice 200, and the use of the information processing device 300. Astream transmission method is different from in the embodiment of thepresent technology, but the control unit 240 of the informationprocessing device 200 can also perform the stream transmission controlwith the information processing device 300 based on the controlperformed from the information processing device 300 based on thecapability information regarding the information processing device 200and the radio wave propagation measurement information regarding thecommunication of the stream with the information processing device 300.

The control unit 370 of the information processing device 300 performsthe control such that the power consumption mode is set in theinformation processing device 200 based on the capability information(for example, the information indicating whether the device is a mobiledevice) regarding the information processing device 200. In this case,the control unit 370 can perform the control such that the low powerconsumption mode is set in the information processing device 200 basedon the capability information regarding the information processingdevice 200 and the management information for managing the informationprocessing device 200. The control unit 240 of the informationprocessing device 200 sets the power consumption mode based on thecontrol performed from the information processing device 300 based onthe capability information regarding the information processing device200. In the embodiment of the present technology, the example of thetopology in which two source devices are used has been described, but anembodiment of the present technology is not limited thereto. Forexample, when the number of devices is 2 or more, it is necessary tocontrol data transmission speed control corresponding to the number ofdevices and state transition is considerable. Therefore, the control isdifficult, but benefit can be obtained. It is possible to alsocorrespond to topology in which two or more source devices areconnected.

[Communication Example Between Source Device and Sink Device Using WebSocket]

Next, an example in which each piece of information is interchangedbetween the source device and the sink device using the Web Socket willbe described.

Here, the WebSocket is a communication standard in which if a connectionis established between a server and a client even once, a datainterchange can be performed through socket communication without beingconscious of a communication process unless explicitly disconnected. Inthe WebSocket, the server and all the clients between which theconnection is established can share data and perform transmission andreception in real time. In other words, in the WebSocket, it is possibleto maintain the port, and it is possible to image transmission and audiotransmission in real time.

Here, in the Wi-Fi CERTIFIED Miracast, there is the UIBC from the sinkdevice to the source device as described above. As in control from thesource device to the sink device, a request related to a control line isassumed to be increased in a topology environment of multiple sources ormultiple sinks.

In this regard, in the first embodiment of the present technology, eachpiece of information is interchanged between the source device and thesink device using the WebSocket. For example, in the CapabilityNegotiation, it is possible to interchange each piece of information forsetting the WebSocket between the source device and the sink device andperform a new additional setting for the WebSocket.

[Example of Setting UIBC and WebSocket Separately]

First, an example of setting the UIBC and the WebSocket separately willbe described.

[Setting Example of UIBC]

FIGS. 20 and 21 are sequence charts illustrating a communication processexample between the source device 410 and the sink device 420 accordingto the first embodiment of the present technology. FIGS. 20 and 21illustrate a communication example (a setting example of the UIBC) forsetting the UIBC in the Capability Negotiation. The UIBC setting methodis specified in the Wi-Fi CERTIFIED Miracast standard.

When the UIBC is set, Wfd-uibc-Capability is interchanged between thesource device 410 and the sink device 420 through an M3 message (RTSPGET_PARAMETER). The purpose of this interchange is to enable the sourcedevice 410 to confirm whether or not the sink device 420 has a UIBCfunction therein.

As illustrated in a of FIG. 20, the source device 410 receives thecapability information of the sink device 420 from the sink device 420,and transmits a UIBC line setting request to the sink device 420 throughan M4 message. Instead of transmitting the UIBC line setting requestthrough the M4 message, the connection request may be transmittedthrough an M14 message after the image transmission starts. When the M14message is transmitted, the request from the source device 410 but alsothe UIBC line setting request from the sink device 420 to the sourcedevice 410 may be performed. This example is illustrated in a of FIG.21.

The UIBC is hardly enabled only through the interchange of the M4/M14message. For this reason, as illustrated in b of FIG. 20 and b of FIG.21, an M15 message is interchanged after a response to the UIBC linesetting request is received. Thus, the UIBC line is enabled.

[Setting Example of WebSocket]

FIGS. 22 and 23 are sequence charts illustrating a communication processexample between the source device 410 and the sink device 420 accordingto the first embodiment of the present technology. FIGS. 22 and 23illustrate a communication example (a setting example of the WebSocket)for setting the WebSocket in the Capability Negotiation.

When the WebSocket is set, Wfd-websocket-Capability is interchangedbetween the source device 410 and the sink device 420 through the M3message (RTSP GET_PARAMETER). The purpose of this interchange is toenable the source device 410 to confirm whether or the sink device 420has a Web Socket function therein.

As illustrated in a of FIG. 22, the source device 410 receives thecapability information of the sink device 420 from the sink device 420,and transmits a WebSocket line setting request to the sink device 420through the M4 message. Instead of transmitting the Web Socket linesetting request through the M4 message, the connection request may betransmitted through the M14 message after the image transmission starts.When the M14 message is transmitted, the request from the source device410 but also the WebSocket line setting request from the sink device 420to the source device 410 may be performed. This example is illustratedin a of FIG. 23.

The WebSocket is hardly enabled only through the interchange of theM4/M14 message. For this reason, as illustrated in b of FIG. 22 and b ofFIG. 23, the M15 message is interchanged after a response to theWebSocket line setting request is received. Thus, the WebSocket line isenabled.

[Setting Example of UIBC and WebSocket]

FIG. 24 is a sequence chart illustrating a communication process examplebetween the source device 410 and the sink device 420 according to thefirst embodiment of the present technology. FIG. 24 illustrates acommunication example (a setting example of the UIBC and the Web Socket)for setting the UIBC and the Web Socket in the Capability Negotiation.

In other words, FIG. 24 illustrates a communication example in which theUIBC connection request/response illustrated in FIGS. 20 and 21 and theWeb Socket connection request/response illustrated in FIGS. 22 and 23are performed. For example, wfd-websocket-Capability is defined as theWebSocket line, and the Capability Negotiation as to whether or not theWebSocket line is implemented is performed between the source device 410and the sink device 420.

Further, FIG. 24 illustrates an example in which a port number for theWebSocket is set to a different port number from that for the UIBC. Asdescribed above, by using a different port for the Web Socket, it ispossible to use the connection links of both the UIBC and the WebSocket,and it is possible to perform a process having a high real-timeproperty.

Further, FIG. 24 illustrates an example in which, when a new command andthe UIBC are designated as different ports on the TCP, the new commandand the UIBC interchange real-time data, and thus the UIBC and theWebSocket are determined as different categories.

Specifically, Wfd-uibc-Capability is interchanged between the sourcedevice 410 and the sink device 420 through the M3 message(Req.GET_PARAMETER_REQUEST). Further, Wfd-websocket-Capability isinterchanged between the source device 410 and the sink device 420through the M3 message (Req.GET_PARAMETER_REQUEST). The purpose of thisinterchange is to enable the source device 410 to confirm whether thesink device 420 has the UIBC function and the Web Socket functiontherein.

The capability of the UIBC and the capability of the WebSocket aretransmitted from the sink device 420 to the source device 410 throughthe M3 message (Res. GET_PARAMETER_RESPONSE). For example, when “GENERIC(coordinates)” is designated for the UIBC, the following content istransmitted:

wfd_uibc_capability: input_category_list = GENERIC; generic_cap_list =Mouse, SingleTouch; hidc_cap_list = none; port = none

Further, for example, when “HIDC (USB)” is designated for the UIBC, thefollowing content is transmitted:

wfd_uibc_capability: input_category_list = HIDC; generic_cap_list =none; hidc_cap_list = Mouse/BT, RemoteControl/Infrared; port = none

For example, for the WebSocket, the following content is transmitted:

wfd-websocket-capability = “wfd_websocket_capability:” SP (“none” /(input-category-val “;xxxx;” tcp-port)) CRLF; “none” if not supported

Further, information for designating the UIBC and the WebSocket istransmitted from the source device 410 to the sink device 420 throughthe M4/M14 message (Req.GET_PARAMETER_REQUEST). For example, when“GENERIC (coordinates)” is designated for the UIBC, the followingcontent is transmitted. An example of a frame format when the content istransmitted is illustrated in FIG. 25.

wfd_uibc_capability: input_category_list = GENERIC; generic_cap_list =Mouse, SingleTouch; hidc_cap_list = none; port = 1000 wfd_uibc_setting:enable

For example, when “ROC (USB)” is designated for the UIBC, the followingcontent is transmitted:

wfd_uibc_capability: input_category_list = HIDC; generic_cap_list =none; hidc_cap_list = Mouse/BT, RemoteControl/Infrared; port = 1000wfd_uibc_setting: enable

For example, for the WebSocket, the following content is transmitted:

wfd-websocket-capability: connection control terminal function; xxxx;port=8000 Wfd_websocket_setting: enable

Here, for example, a CTRL (corresponding to (B1) to (B3) described inthe second embodiment of the present technology, for example) isdescribed in the “connection control function.”

FIG. 25 illustrates an example of a frame format of the UIBC transmittedfrom the source device 410 to the sink device 420 through the M4/M14message (Req.SET_PARAMETER_REQUEST).

[Frame Format Example of UIBC]

FIG. 25 illustrates an example of the frame format of the UIBCinterchanged between the source device 410 and the sink device 420according to the first embodiment of the present technology. FIG. 25illustrates an example of the frame format of the UIBC when a GenericInput is designated.

b of FIG. 25 illustrates content stored in a field “Generic Input TypeID” illustrated in a of FIG. 25.

c of FIG. 25 illustrates content stored in a field “Describe” of a ofFIG. 25.

In the above example, the same method as the connection method of theUIBC is used as the connection method of the WebSocket, but a differentmethod from the connection method of the UIBC may be used as theconnection method of the WebSocket. Further, the example using adefinition of the attribute such as wfd-WebSocket-Capability has beendescribed, but the present technology is not limited to this example.For example, a different attribute may be used as long as it has theequivalent function.

In the example illustrated in FIG. 24, the different port number fromthe port number for the UIBC is set as the port number for the WebSocket, but the same port may be set for the UIBC and the WebSocket. Inthis regard, FIG. 26 illustrates a connection example in which the UIBCand the WebSocket are operated through the same port.

[Setting Example of UIBC and WebSocket when Same Port is Set for UIBCand Web Socket]

FIG. 26 is a sequence chart illustrating a communication process examplebetween the source device 410 and the sink device 420 according to thefirst embodiment of the present technology. FIG. 26 illustrates acommunication example (a setting example of the UIBC and the Web Socket)for setting the UIBC and the Web Socket in the Capability Negotiation.

For example, it can be implemented by extending the Wi-Fi CERTIFIEDMiracast standard. For example, the “WebSocket” is added to a categorysuch as input-cat of the M3 message (Req.GET_PARAMETER_REQUEST). Forexample, the sink device 420 may select the GENERIC among the GENERIC,the HIDC, and the Web Socket.

In this case, for example, the following content is transmitted from thesink device 420 to the source device 410 as the M3 message(Res.GET_PARAMETER_RESPONSE):

wfd_uibc_capability: input_category_list = WebSocket; generic_cap_list =connection control function; hidc_cap_list = none; port = none

In this example, the example corresponding to one category has beendescribed, but it is possible to correspond to two or more categories atthe same time.

For example, the following content is transmitted from the source device410 to the sink device 420 as the M4/M14 message(Req.SET_PARAMETER_REQUEST):

wfd_uibc_capability: input_category_list = WebSocket; generic_cap_list =connection control function; hidc_cap_list = none; port = 1000wfd_uibc_setting: enable

As described above, it is possible to designate the same port for theUIBC and the Web Socket on the TCP.

In this example, the example corresponding to one category has beendescribed, but it is possible to correspond to two or more categories atthe same time.

The example in which the WebSocket is added to input-cat when the sameport is set has been described, but the present technology is notlimited to this example. For example, it may be set as any othercategory. The Capability Negotiation may be performed twice instead ofsetting two input-cats at the same time.

The example in which the connection method is changed according towhether the same port is set for the UIBC and the Web Socket or thedifferent ports are set for the UIBC and the WebSocket has beendescribed above. However, switching from the same port to the differentports (or the different ports to the same port) may be performed in themiddle of the connection (or after the connection). For example, afterthe Capability Negotiation of the same port is performed in the UIBC andthe Web Socket, switching to the different ports may be performed whenthe usage frequency of the UIBC is high. FIG. 27 illustrates an exampleof switching the port as described above.

[Operation Example of Information Processing Device (Sink Device)]

FIG. 27 is a flowchart illustrating an example of a processing procedureof a port setting process performed by the information processing device300 according to the first embodiment of the present technology.

First, the control unit 370 of the information processing device 300starts the Capability Negotiation with the source device (step S801).Then, the control unit 370 determines whether or not the request for theUIBC and the WebSocket has been received from the source device (stepS802). When the request has not been receives (step S802), the operationof the port setting process ends.

When the request for the UIBC and the WebSocket has been received fromthe source device (step S802), the control unit 370 confirms whether ornot the same port is set for the UIBC and the WebSocket based on thedisplay region of the display unit 351 (step S803).

When there is no problem even though the same port is set for the UIBCand the WebSocket (step S804), the control unit 370 performs aconnection control setting based on the same port (step S805). Forexample, the interchange for setting the same port for the UIBC and theWebSocket is performed as illustrated in FIG. 26.

When a problem occurs if the same port is set for the UIBC and theWebSocket (step S804), the control unit 370 performs the connectioncontrol setting based on the different ports (step S806). For example,the interchange for setting the different ports for the UIBC and the WebSocket is performed as illustrated in FIG. 24.

Then, the control unit 370 changes the display form of the display unit351 for displaying the image transmitted from the source device (stepS807). Then, after changing the display form, the control unit 370determines whether or not there is a source device whose port settinghas to be changed (step S808). When there is no source device whose portsetting has to be changed (step S808), the operation of the port settingprocess ends.

When there is a source device whose port setting has to be changed (stepS808), the control unit 370 performs control such that the differentports are set for the source devices (in which the UIBC and theWebSocket are set to the same port) (step S809).

Then, the control unit 370 determines whether or not the number of portshas to be reduced (step S810). When the number of ports need not bereduced (step S810), the operation of the port setting process ends.

When the number of ports has to be reduced (step S810), the control unit370 performs control such that the same port is set for the sourcedevices in which the UIBC and the WebSocket are set to the differentports (step S811). Steps S802 to S811 is an example of a control processset forth in claims.

As described above, the wireless communication unit 320 of theinformation processing device 300 performs the real-time imagetransmission with the source device according to the Wi-Fi CERTIFIEDMiracast specification. The control unit 370 performs control such thatthe image based on the image information transmitted from the sourcedevice is displayed on the display unit 351, and the port used in aplurality of communication schemes (the UIBC and the Web Socket) forinterchanging the control information related to the image with thesource device is set. In this case, for example, when the port is set onthe TCP session generated between the information processing device 300and the source device, the control unit 370 transmits the information(for example, the information described above with reference to FIGS. 24and 25) for designating a plurality of communication schemes using theport to the source device.

The control unit 370 can set a plurality of ports for a plurality ofcommunication schemes as the port used for a plurality of communicationschemes. In this case, the control unit 370 can set a plurality of portsaccording to the request transmitted from the source device.

Further, the control unit 370 can set one port for a plurality ofcommunication schemes as the port used for a plurality of communicationschemes.

The control unit 370 can transmit the information related to the displayform of the image of the source device in the display unit 351 to thesource device using any one of a plurality of communication schemes andchange the display form of the display based on the control informationtransmitted from the source device. The control information is, forexample, the manipulation information related to the manipulation forchanging the display form of the source device image received in thesource device.

The source device side can perform the same operation. For example, thecontrol unit 240 of the information processing device 200 transmits theimage information for causing the image to be displayed on the displayunit of the sink device side to the sink device. The control unit 240can perform control such that the port used for a plurality ofcommunication schemes for interchanging the control information relatedto the image with the sink device is set.

Here, for example, when different ports are set for the UIBC and theWebSocket, it is necessary to set two different links for each sourcedevice. In other words, the number of ports of the source device thatare necessary is increased to be twice. For this reason, for example,when several tens of thousands of source devices are connected to thesink device, it may be hard to allocate the ports. In this case, thesystem response is likely to get worse. In this regard, in the firstembodiment of the present technology, one or more ports areappropriately set according to the usage environment.

The sink device can switch the control channel of the UIBC to the sourcedevice that transmits the request in the middle of connection totransmit the information related to the display position of the sinkdevice based on the request transmitted from the source device. Thesource device can switch the control channel of the UIBC to the sinkdevice that transmits the request in the middle of connection totransmit the display position information of the source device based onthe manipulation transmitted from the sink device.

Here, it is possible to transmit and receive image data or audio data aswell as a text using the control line of the Web Socket. In this regard,an example in which image data or audio data are transmitted andreceived using the control line of the WebSocket will be describedbelow.

[Transmission Example of Image Data and Audio Data Using WebSocket]

FIG. 28 is a diagram illustrating a communication example of image dataor audio data using the WebSocket between the source device 410 and thesink device 420 according to the first embodiment of the presenttechnology. a of FIG. 28 illustrates a communication process examplebetween the source device 410 and the sink device 420. b and c of FIG.28 illustrates an example of a frame format of the WebSocketinterchanged between the source device 410 and the sink device 420.

In the example illustrated in FIG. 28, the source device 410 of theWi-Fi CERTIFIED Miracast functions as a client of the WebSocket, and thesink device 420 functions as a server of the Web Socket.

For example, the source device 410 transmits Host, Sec-WebSocket-Key,and Sec-WebSocket-Version to the sink device 420 (441). Host is used todesignate a connection destination of the WebSocket. Sec-WebSocket-Keyis used to obtain a handshake response. Sec-WebSocket-Version is used todesignate a version of the WebSocket.

The sink device 420 generates a value of Sec-WebSocket-Accept based on avalue of Sec-WebSocket-Key received from the source device 410. Then,the sink device 420 transmits the generated Sec-WebSocket-Accept to thesource device 410 (442). Here, Sec-WebSocket-Accept is used to maintaina connection with the client.

The source device 410 transmits the frame format illustrated in b ofFIG. 28 to the sink device (441). In the frame format, data serving as atransmission target can be designated by a WebSocket Type ID. An exampleof the Web Socket Type ID is illustrated in c of FIG. 28.

Further, image data or audio data is interchanged between the sourcedevice 410 and the sink device 420 as binary data of the WebSocket(443).

For example, when FIN is 0×0, it indicates that the payload iscontinued, and thus the sink device 420 performs connection withimmediately previous data. Further, when FIN is 0×1, it is a flagindicating the end of the payload, and thus the sink device 420processes the payload.

For example, RSV is an extension bit.

For example, when opcode is 0×0, it indicates that the payload iscontinued, and the sink device 420 performs connection with immediatelyprevious data. Further, when opcode is 0×1, it indicates that thepayload is a text, and when opcode is 0×2, it indicates that the payloadis binary.

For example, MASK is a bit indicating whether or not a message is anencoded message. The message from the source device 410 to the sinkdevice 420 is necessarily an encoded message.

As described above, when a binary value of opcode is used, image data oraudio data can be interchanged between the source device 410 and thesink device 420 as the binary data of the WebSocket. In other words,image data and audio data can be transmitted using the Web Socketstandard format. Thus, prior authentication can be performed using imagedata or audio data. Further, it is possible to correspond to variouskinds of other use cases.

The image data and audio data can be performed between the source deviceand the sink device using the Web Socket while performing the imagetransmission according to the Wi-Fi CERTIFIED Miracast standard.

As described above, the control unit 370 of the information processingdevice 300 can acquire the image information and the audio informationtransmitted from the source device using the WebSocket. The control unit370 can cause an image based on the image information to be displayed onthe display unit 351 and cause a sound based on the audio information tobe output from the audio output unit 352.

As described above, in the first embodiment of the present technology,the sink device can perform connection control terminal management ofall the source devices and receive the connection control manipulationfrom all the source devices. For example, it is possible to correspondeven to an environment in which all the source devices are prohibitedfrom entering the image transmission mode due to limitations oftransmission bands. In other words, in an environment in which imagestransmitted from a plurality of source devices are displayed on thedisplay unit of the sink device side, it is possible to stably operatethe image transmission mode and performing a real-time connectioncontrol manipulation.

When images transmitted from a plurality of source devices are displayedone sink device, it is possible to notify the source device of thedisplay positions of the images in the sink device in real time, and thesource device can control the display position. Further, it is possibleto interchange the information related to the display position using theUIBC and the WebSocket.

Thus, by reducing useless processes of the source device and performingprocesses in the sink device, the load of the source device can bereduced. It is possible to perform a process having a high real-timeproperty by using the connection links of both the UIBC and theWebSocket. For example, even in the environment in which the number ofsource devices is large, it is possible to reduce the load of the sinkdevice and perform the position information process and the UIBCtransmission process.

As described above, the information related to the source device and theimage of the source device can be displayed on the display unit of thesink device side in association with each other. In this case, it ispossible to perform a manipulation in the order in which the request isreceived according to the request transmitted from the source device.

2. SECOND EMBODIMENT

In a second embodiment of the present technology, an example in whichthe connection between the source device and the sink device via theaccess point and the direct connection between the source device and thesink device are switched (or simultaneously used) will be described.

[Configuration Example of Communication System]

FIG. 29 is a diagram illustrating a system configuration example of acommunication system 600 according to the second embodiment of thepresent technology.

The communication system 600 includes an access point 601, a network602, information processing devices 603, 604, 610, 620, and 630, adisplay device 631, and a control device 640.

The access point 601 is an access point of a wireless LAN (for example,Wi-Fi). For example, the access point 601 has a function of aninfrastructure mode of an IEEE802.11 standard. The access point 601 isconnected to one or more information processing devices (for example, atransmission side information processing device (source device) and areception side information processing device (sink device)).

The access point 601 can be connected with the information processingdevice 630 via a wired line (for example, Ethernet (registeredtrademark)). For example, the access point 601 can be connected with theinformation processing device 630 via the network 602. The access point601 may be connected with an internal bus of the information processingdevice 630 and perform a process. Examples of the internal bus of theinformation processing device 630 include a Universal Serial Bus (USB),Peripheral Component Interconnect (PCI), and PCI Express. The connectionbetween the access point 601 and the information processing device 630may be a wired connection or a wireless connection (for example, thewireless LAN). For example, in the case of the wireless LAN, the network602 is connected to the information processing device 630, and theinformation processing device 630 has to determine whether atransmission/reception process is a transmission/reception process withthe access point 601 or a transmission/reception process with theinformation processing device 610.

In the second embodiment of the present technology, an example in whichthe access point 601 and the information processing devices 603, 604,610, and 620 are connected using the wireless LAN (for example, thewireless LAN corresponding to IEEE 802.11 a/b/g/n/ac/ad) will bedescribed. An example in which the access point 601 and the informationprocessing device 630 are connected via an Ethernet line will bedescribed.

Further, an example in which the information processing devices 603,604, 610, and 620 are assumed to be the transmission side informationprocessing device (source device), and the information processing device630 is assumed to be the reception side information processing device(sink device) will be described. The information processing devices 603,604, 610, and 620 correspond to the information processing device 200illustrated in FIG. 2, and the information processing device 630corresponds to the information processing device 300 illustrated in FIG.3. In the following description, the information processing device 610will be mainly described as the source device, and the descriptionsimilarly applies to the information processing devices 603, 604, and620.

The information processing device 610 is the transmission sideinformation processing device (the source device) that transmits animage. The information processing device 610 is an informationprocessing device that can be connected with the access point 601 in theinfrastructure mode.

Here, a connection example in which the information processing device610 establishes the connection with the access point 601 in theinfrastructure mode will be described. The information processing device610 transmits a connection request signal to the access point 601 inresponse to a beacon signal transmitted from the access point 601. Asdescribed above, the information processing device 610 starts anoperation (a connection establishment operation) for establishing theconnection with the access point 601 by transmitting the connectionrequest signal.

The connection establishment operation is an operation of linking alevel of a layer 2. For example, a connection authentication process ofdetermining whether or not the information processing device 610 isconnected as an application is necessary in addition to passwordauthentication using packet identification (PID) performed in thewireless LAN.

The information processing device 610 can establish the connection withthe information processing device 630 through P2P direct communication(for example, WiFi Direct). For example, the information processingdevice 610 has a protocol capable of establishing the connection withthe information processing device 630 through the WiFi Direct andenabling image communication such as the Wi-Fi CERTIFIED Miracast ordigital living network alliance (DLNA).

The information processing device 610 may be a low-speed image and audiotransmission device (a standby device) that establishes the connectionwith the access point 601 and interchanges various kinds of informationwith the information processing device 630 via the access point 601.Further, when the information processing device 610 is used only as thelow-speed image and audio transmission device, the informationprocessing device 610 may not have a protocol capable of enablinghigh-speed image audio transmission.

The information processing device 630 has a communication function forestablishing a connection with the access point 601, the informationprocessing devices 603, 604, 610, and 620, or the control device 640.The information processing device 630 has a function of establishing theconnection with the access point 601, the information processing devices603, 604, 610, and 620, and the control device 640 and controlling allprotocols. The information processing device 630 has a communicationfunction of establishing a connection with a private LAN or a globalLAN.

Here, it is important to stabilize the connection between theinformation processing device 630 and the information processing devices603, 604, 610, and 620. Thus, it is desirable to use a frequencydifferent from a radio frequency used by the access point 601 for theconnection between the information processing device 630 and theinformation processing devices 603, 604, 610, and 620. In the secondembodiment of the present technology, an example in which a frequencyband such as 2.4 GHz, 5 GHz, or 60 GHz of IEEE 802.11 a/b/g/n/ac/ad isused will be described. Here, any other wireless standard or any otherfrequency band may be used.

The information processing device 630 has a display function ofdisplaying an image. The display function is a function of performingscreen display of a television or a projector. The informationprocessing device 630 may cause an image to be displayed on a displayunit with which the information processing device 630 is equipped or maycause an image to be displayed on an external display device. In thesecond embodiment of the present technology, an example in which thedisplay device 631 separate from the information processing device 630is installed will be described.

When the display device 631 separate from the information processingdevice 630 is installed as described above, the information processingdevice 630 and the display device 631 can be connected through wiredconnection or wireless connection. For example, when the informationprocessing device 630 and the display device 631 are connected throughwired connection, the connection can be established using a wirelinecable (for example, High-Definition Multimedia Interface (HDMI)(registered trademark), Mobile High-Definition Link (MEL), DisplayPort,or USB3.0). Further, for example, when the information processing device630 and the display device 631 are connected through wirelessconnection, the connection can be established using the wireless LAN. Aplurality of display devices may be connected to the informationprocessing device 630.

The control device 640 is a connection control device that manages theinformation processing devices (the information processing devices 603,604, 610, 620, and 630) included in the communication system 600, anddetects and controls the connection of the information processingdevices and a display state. For example, the control device 640 managesand detects information (for example, terminal identificationinformation and capability information) related to the informationprocessing devices and an operation state (for example, a communicationmode and a group participation state) of the information processingdevices. The control device 640 is connected with the informationprocessing device 630 using wireless communication. For example, acommunication scheme (for example, the wireless LAN) supported by theinformation processing device 630 may be used as the wirelesscommunication. An infrared scheme, any other wireless scheme, or a wiredline may be used. The control device 640 may have a protocol forestablishing the connection with the access point 601 and establish theconnection with the information processing device 630 via the accesspoint 601.

For example, the control device 640 can manage the informationprocessing devices by detecting whether or not the informationprocessing device is participating in the group or has left the group.Each of the source devices participating in the group performs, forexample, a process of transmitting a command to the access point 601 orthe information processing device 630 at a level at which powerconsumption is not influenced. When the command is received directly orindirectly via the access point 601, the information processing device630 notifies the control device 640 of information related to the sourcedevice from which the command has been received. Thus, the controldevice 640 can confirm whether or not the source device has left thegroup.

For example, the control device 640 causes an image (for example, anicon) corresponding to the source device participating in the group tobe displayed on an input and output unit 643. For example, the controldevice 640 confirms whether or not the source device participating inthe group has left the group, and when the source device that has leftthe group is detected, the control device 640 deletes the image (forexample, the icon) corresponding to the source device that has left thegroup from the input and output unit 643. For example, when a commandtransmitted from a certain information processing device is determinedto have not received during a predetermined period of time, the controldevice 640 determines the information processing device to have left thegroup. Then, the control device 640 deletes the image (for example, theicon) corresponding to the information processing device that has leftthe group from the input and output unit 643.

The control device 640 can cause the image (for example, the icon)corresponding to the information processing device participating in thegroup to be displayed on the input and output unit 643 according to aplace where the information processing devices exists. For example,position information (for example, a latitude and a longitude) of eachinformation processing device may be acquired, and an absolute positionof the information processing device may be decided based on theposition information. It is possible to decide relative positions of theinformation processing devices based on the positions of the informationprocessing devices decided as described above and arrange the students.The information processing devices can acquire the position information,for example, using the Global Positioning System (GPS).

For example, the relative position of the information processing devicemay be detected using radio waves. For example, the informationprocessing device 630 may acquire radio wave strengths from a pluralityof information processing devices and acquire a relative position (aposition of the own information processing device) with the otherinformation processing devices based on the radio wave strengths using atriangulation technique. The control device 640 may acquire the relativepositions obtained as described above and arrange the image of theinformation processing device.

An example in which the control device 640 manages one or moreinformation processing devices connected to the information processingdevice 630, and the images indicating the information processing devicesare displayed on the input and output unit 643 of the control device 640is described. However, one or more information processing devicesconnected to the information processing device 630 may be managed by theinformation processing device 630, and the images indicating theinformation processing devices may be displayed on the display unit ofthe information processing device 630 or the display device 631.

[Switching Example of Communication Mode of Source Device]

Here, a switching method when the communication modes (the low-speedimage and audio transmission mode (the standby mode), and the high-speedimage audio transmission mode) of the source device are switched will bedescribed.

For example, the following (B1) to (B3) can be used as triggers forswitching of the communication modes of the source device:

(B1) switching using the source device (for example, active switchingaccording to a manipulation of the user using the source device);

(B2) switching using the control device (for example, switching (remotemanipulation switching) according to a manipulation of the user usingthe control device 640); and

(B3) switching using the sink device (for example, switching accordingto a manipulation of the user using the sink device (or the displaydevice 631 connected thereto)).

The manipulation (the switching manipulation) of the user in each deviceis, for example, the manipulation of the user using the manipulationreception unit (for example, a manipulation member or a touch panel)with which each device is equipped or the manipulation of the user usingthe user information acquisition unit (for example, a detection unitthat detects a gesture) with which each device is equipped.

In (B1), for example, a manipulation input of an application or amanipulation member installed in the source device in advance can beused as a switching trigger. For example, when the switchingmanipulation is performed in the source device, a command related toswitching of the communication mode is transmitted to the sink device(the information processing device 630). When the command is received,the sink device (the information processing device 630) performs controlsuch that the band of the source device that has transmitted the commandis increased or decreased while performing control for the other bands.The sink device (the information processing device 630) transmitsinformation indicating that the communication mode of the source devicethat has transmitted the command has been switched to the control device640. In this case, the control device 640 may output notificationinformation indicating that the communication mode has been switched(through a display or an audio output).

In (B2), for example, the manipulation of the user in the sink device(the information processing device 630) can be used as the switchingtrigger. For example, when the switching manipulation of the sourcedevice is performed in the sink device (the information processingdevice 630), control is performed such that the band of the sourcedevice in which the switching manipulation has been performed isincreased or decreased while performing control for the other bands. Thesink device (the information processing device 630) transmitsinformation indicating that the communication mode of the source devicein which the switching manipulation has been performed has been switchedto the control device 640. In this case, similarly to the case of (B1),the control device 640 may output notification information indicatingthat the communication mode has been switched.

In (B3), for example, the manipulation of the user in the control device640 can be used as the switching trigger. For example, when theswitching manipulation of the source device is performed in the controldevice 640, information (for example, the terminal identificationinformation) related to the source device in which the switchingmanipulation has been performed is transmitted to the sink device (theinformation processing device 630). When the information is received,the sink device (the information processing device 630) performs controlsuch that the band of the source device in which the switchingmanipulation has been performed is increased or decreased whileperforming control for the other bands.

The control device 640 manages device connected to the control device640 as described above. The control device 640 confirms whether or noteach device has left the group, and when there is a device that has leftthe group, the control device 640 deletes the image (for example, theicon) corresponding to the device that has escaped from the control ofits own information processing device from the input and output unit643.

Here, the manipulation of the user in the control device 640 will bedescribed. For example, the control device 640 cause the image (forexample, the icon) corresponding to the information processing deviceparticipating in the group to be displayed on the input and output unit643 so that the image is arranged according to a place where theinformation processing devices exists. In this case, an informationprocessing device that is set to a low-speed image audio transmissionmode is assumed to perform switching to a high-speed image audiotransmission mode (the peripheral channel). In this case, a manipulationof moving an image corresponding to the information processing deviceserving as a switching target to a first region (for example, apresentation region) is performed. For example, the drag & dropmanipulation is performed. Thus, the information processing device thatis set to the low-speed image audio transmission mode can be switched tothe high-speed image audio transmission mode (the peripheral channel).

Further, for example, an information processing device that is set tothe high-speed image audio transmission mode (the middle channel or theperipheral channel) is assumed to perform switching to the low-speedimage audio transmission mode. In this case, a manipulation of moving animage corresponding to the information processing device serving as aswitching target to a second region (for example, a standby region) isperformed. For example, a drag & drop manipulation is performed. Thus,an information processing device that is set to the high-speed imageaudio transmission mode (the middle channel or the peripheral channel)can be switched to the low-speed image audio transmission mode.

Similarly to the cases of (B1) and (B2), the control device 640 mayoutput the notification information indicating that the communicationmode has been switched.

The control device 640 may cause the images indicating the informationprocessing devices to have different display states according to thecommunication mode. For example, the contour of the image indicating theinformation processing device set to the high-speed image audiotransmission mode (the middle channel) is indicated by a thick line, andthe contour of the image indicating the information processing deviceset to the high-speed image audio transmission mode (the peripheralchannel) is indicated by a fine line. The information processing deviceset to the low-speed image audio transmission mode may be indicated by adotted line. As another example, the color of the image indicating theinformation processing device set to the high-speed image audiotransmission mode (the middle channel) is red, and the color of theimage indicating the information processing device set to the high-speedimage audio transmission mode (the peripheral channel) is blue. Further,the color of the image indicating the information processing device setto the low-speed image audio transmission mode may be white.

In the second embodiment of the present technology, a connection schemeof indirectly connecting a source device with a sink device via anaccess point is employed in the low-speed image audio transmission modeas described above. In the high-speed image audio transmission mode, aconnection scheme of directly connecting the source device with the sinkdevice without intervention of the access point is employed.

[Operation Example of Information Processing Device (Source Device)]

FIG. 30 is a flowchart illustrating an example of a processing procedureof the data transmission process performed by the information processingdevice 610 according to the second embodiment of the present technology.

FIG. 30 illustrates an example in which the source device (theinformation processing device 610) is directly connected to the accesspoint and then starts each process as default. In other words, theinformation processing device 610 activates an application that isinstalled in advance, and transmits the connection request to the sinkdevice via a specific access point (the access point 601). Further, FIG.30 illustrates an example in which the source device (the informationprocessing device 610) is connected to the information processing device630 serving as the sink device and connected to the access point 601serving as a specific access point.

First, the information processing device 610 activates an applicationfor performing communication with the sink device (the informationprocessing device 630) (step S821). After the activation, the sinkdevice (the information processing device 630) serving as a connectiontarget is selected (step S821).

Then, the control unit of the information processing device 610transmits the connection request to the information processing device630 via the access point 601 (AP) (step S822). Then, the control unit ofthe information processing device 610 determines whether or not theconnection permission information has been received from the informationprocessing device 630 via the access point 601 (step S823). When theconnection permission information has not been received from theinformation processing device 630 (step S823), the operation of the datatransmission process ends. In this case, the operation of the datatransmission process may end when the connection permission informationhas not been received within a predetermined period of time.

When the connection permission information has been received from theinformation processing device 630 (step S823), the control unit of theinformation processing device 610 transmits the capability informationof the information processing device 610 to the information processingdevice 630 via the access point 601 (step S824).

Then, the control unit of the information processing device 610determines whether or not the setting request information has beenreceived from the information processing device 630 via the access point601 (step S825). When the setting request information has not beenreceived (step S825), it is continuously monitored.

When the setting request information has been received (step S825), inthe information processing device 610, the standby mode is set as theinitial state. Then, the control unit of the information processingdevice 610 determines whether or not the standby mode has been set (stepS826). Then, when the standby mode has been set in the informationprocessing device 610 (step S826), it is continuously monitored.

When the standby mode has not been set (when the image transmission modehas been set) (step S826), the control unit of the informationprocessing device 610 determines whether or not the informationprocessing device 610 has left the group (step S827). Here, when theinformation processing device 610 has left the group, for example, whenthe information processing device 610 has moved to a place that neitherthe radio waves of the access point nor the sink device reaches, amanipulation of the user for leaving the group may be performed. Thismanipulation of the user may be, for example, a manipulation of pressinga leaving button (for example, a manipulation of pressing themanipulation member or a manipulation of pressing the touch panel). Themanipulation of the user is assumed to be performed in any one of theinformation processing device 610, another source device, the sinkdevice, and the control device 640.

Then, when the information processing device 610 has left the group(step S827), the operation of the data transmission process ends. Whenthe information processing device 610 has not left the group (stepS827), the control unit of the information processing device 610disconnects the connection with the information processing device 630via the access point 601 (step S828).

Then, the control unit of the information processing device 610 performsthe P2P connection process for performing the direct connection with theinformation processing device 630 (step S829). After the P2P connectionprocess is completed (step S829), the control unit of the informationprocessing device 610 transmits the connection request to theinformation processing device 630 based on the setting requestinformation received from the information processing device 630 (stepS830).

Then, the control unit of the information processing device 610determines whether or not the confirmation information indicatingpermission for reception of an image corresponding to the connectionrequest has been transmitted from the information processing device 630(step S831). When the confirmation information has not been transmittedfrom the information processing device 630 (step S831), it iscontinuously monitored.

When the confirmation information has been transmitted from theinformation processing device 630 (step S831), the control unit of theinformation processing device 610 starts the direct P2P connection withthe information processing device 630 (step S832). Then, the controlunit of the information processing device 610 performs the negotiationof the image parameter and the audio parameter with the informationprocessing device 630 based on the connection request corresponding tothe received confirmation information, and transmits the image data(step S833).

Then, the control unit of the information processing device 610determines whether or not the image transmission mode has been set (stepS834). Then, when the image transmission mode has been set in theinformation processing device 610 (step S834), the process returns tostep S833.

When the image transmission mode has not been set (when the standby modehas been set) (step S834), the control unit of the informationprocessing device 610 determines whether or not the line has beendisconnected (step S835). For example, when the information processingdevice 610 has left the group, the line with the access point 601 or theinformation processing device 630 is disconnected. Then, when the linehas been disconnected (step S835), the operation of the datatransmission process ends.

When the line is not disconnected (step S835), the control unit of theinformation processing device 610 disconnects the direct connection withthe information processing device 630 (step S836). Then, the controlunit of the information processing device 610 starts the connection withthe access point 601 (step S837).

As described above, the information processing device 610 performsreal-time image transmission with the information processing device 630according to the Wi-Fi CERTIFIED Miracast specification. The controlunit of the information processing device 610 performs control such thatsetting request information for causing the information processingdevice 610 to perform a setting related to the real-time imagetransmission is received from the information processing device 630 viathe access point 601. Further, the control unit of the informationprocessing device 610 performs control such that a setting request forperforming a setting based on the setting request information istransmitted to the information processing device 630 through the directcommunication with the information processing device 630. The settingrequest information is generated based on the capability information(related to the information processing device 610) interchanged betweenthe information processing device 610 and the information processingdevice 630 via the access point 601.

Further, when the connection state is switched, the control unit of theinformation processing device 610 can perform control such that theconnection process is performed using at least one of port informationand IP information related to a connection before the switching. Theswitching of the connection state is switching from the connection statewith the information processing device 630 via the access point 601 tothe connection state with the information processing device 630 by thedirect communication or inverse switching.

In the image transmission mode (the peripheral channel), the controlunit of the information processing device 610 performs control such thatan image to be displayed on the peripheral channel region in the displaydevice 631 is transmitted through the wireless transmission path of thelow frequency band. In the image transmission mode (the peripheralchannel), the control unit of the information processing device 610performs control such that an image to be displayed on the middlechannel region in the display device 631 is transmitted through thewireless transmission path of the high frequency band.

The control unit of the information processing device 610 performscontrol such that the setting request is transmitted to the informationprocessing device 630 through the direct communication based on themanipulation of the user in any one of the devices. Alternatively, thecontrol unit of the information processing device 610 performs controlsuch that the setting request is transmitted to the informationprocessing device 630 through the direct communication in apredetermined order (for example, the order of images displayed on theperipheral channel region side by side).

The control unit of the information processing device 630 performscontrol such that the setting request information for causing theinformation processing device 610 to perform the setting related to thereal-time image transmission is transmitted to the informationprocessing device 610 via the access point 601. Further, the controlunit of the information processing device 630 performs control such thatthe setting request for performing a setting based on the settingrequest information is received from the information processing device610 through the direct communication with the information processingdevice 610.

When switching between the access point and the direct connection isperformed as described above, the switching period of time can bereduced. For example, in an environment in which the band control isperformed on a plurality of source devices, an Infrastructureenvironment, and a WiFi Direct (or the Wi-Fi CERTIFIED Miracast)environment, the switching period of time can be reduced. Thus, theseamless connection switching can be provided.

In a multi-source environment, it is possible to set a connection orderand share image information of a high-quality image while causing aplurality of source devices to participate in the group. For example, itis possible to notify the sink device (or the control device) of thesource device that participates in an Infrastructure environment. In theWiFi Direct (or Wi-Fi CERTIFIED Miracast) environment, it is possible toperform the image transmission through the direct communication througha frequency channel different from that in the Infrastructureenvironment. Thus, even when a plurality of source devices are connectedto a single display device (sink device), stable image communication canbe performed. Further, even when hardly any of the source devices areswitched to the image transmission mode due to wireless bandrestrictions, all connections can be stably established.

As the number of connected devices in the group increases, the overheadof connection control increases, band usage efficiency gets worse, andthus it is difficult to perform image transmission stably. On the otherhand, in the embodiment of the present technology, it is possible toprovide the communication system capable of performing stable imagetransmission even in the environment in which the number of connecteddevices is large. In other words, in the embodiment of the presenttechnology, it is possible to perform appropriate control whencommunication is performed between a plurality of information processingdevices.

Here, the sink device can include a server of the WebSocket therein andperform a state management of connected devices. For example, when thestandby mode is set, the WebSocket can be operated via the access pointthrough a connection line for the Web Socket necessary for the statemanagement of connected devices. Further, when the image transmissionmode is set, the WebSocket can be operated on the Wi-Fi CERTIFIEDMiracast. Through theses operations, even when the client of theWebSocket performs switching such as (1) to (3) described above,switching information is transferred to the sink device in real time.The sink device can return the management result to the client (thesource device) of the WebSocket in real time.

[Example in which Connection with Public Line Other than InitiallyConnected Access Point can be Established]

In the above example, when the source device is connected to the sinkdevice, the connection with the sink device via the access point or thedirect connection with the sink device is performed. Here, the userusing the source device may move when the source device is a portableinformation processing device (for example, a smartphone or a tabletterminal). In this case, for example, the user using the source devicemay also move to a place in which it is difficult to establish theconnection with the access point. In this case, the user may desire tocontinuously perform the connection with the sink device. In thisregard, an example in which the user using the source device moves to aplace in which it is difficult to establish the connection with theaccess point will be described below.

[Operation Example of Information Processing Device (Source Device)]

Each of FIGS. 31 and 32 is a flowchart illustrating an example of aprocessing procedure of a data transmission process performed by theinformation processing device 610 according to the second embodiment ofthe present technology.

[Operation Example of Information Processing Device (Source Device)]

FIGS. 31 and 32 illustrate an example in which, as a default, after thesource device (the information processing device 610) is directlyconnected to the access point, the source device (the informationprocessing device 710) starts each process and can also be connected toa public line other than the access point. The example illustrated inFIGS. 31 and 32 is a modified example of FIG. 30, and a description ofparts that are common to FIG. 30 is partially omitted.

First, an application is activated (step S841), and the connectionrequest is transmitted to the information processing device 630 via theaccess point 601 (AP) (step S842). Then, the control unit of theinformation processing device 610 determines whether or not theconnection permission information has been received from the informationprocessing device 630 via the access point 601 (step S843). When theconnection permission information has not been received from theinformation processing device 630 (step S823), the operation of the datatransmission process ends. Steps S841 to S843 correspond to steps S821to S823 illustrated in FIG. 30.

When the connection permission information has been received from theinformation processing device 630 (step S843), the control unit of theinformation processing device 610 determines whether or not information(outdoor use information) for giving permission for outdoor use isincluded in the connection permission information (step S844). Theoutdoor use information is permission information indicating whether ornot the connection with the information processing device 630 via theaccess point other than the access point 601 or the base station ispermitted. Further, when the information processing device 630 transmitsthe connection permission information in response to the connectionrequest received via the access point 601 (AP), the informationprocessing device 630 acquires the outdoor use information (permissionor non-permission) from the control device 640, includes the acquiredoutdoor use information (permission or non-permission) in the connectionpermission information, and transmits the resulting connectionpermission information. Further, although “permission” is stored as theoutdoor use information, when it is a time zone other than a time zoneset as the available group participation time, the informationprocessing device 630 includes the outdoor use information of thenon-permission in the connection permission information and transmitsthe resulting connection permission information. In other words, thecontrol device 640 can perform scheduling (schedule management), managescheduled participation times, and permit the outdoor use only in thetime zone. The outdoor use information may be included in the capabilityinformation and transmitted from the sink device to the source device.

Further, when the outdoor use information of the permission is includedin the connection permission information and transmitted, theinformation processing device 630 includes setting information for usingthe public network in a place that the radio waves of the access point601 do not reach in the connection permission information.

In this example, the permission or the non-permission for the outdooruse is set in advance, but the permission or the non-permission may beset in each device each time there is the connection request from thesource device. For example, the manipulation for setting the permissionor the non-permission can be performed using at least one of the othersource devices (for example, the information processing devices 603,604, and 620), the sink device (the information processing device 630),and the control device 640.

For example, the information processing device 630 can cause a displayscreen for selecting the permission or the non-permission to bedisplayed on the input and output unit 643 of the control device 640 andreceive a selection manipulation of the permission or the non-permissionperformed on the display screen.

Further, for example, the information processing device 630 can causethe display screen for selecting the permission or the non-permission tobe displayed on the display unit 632 of the display device 631 andreceive the selection manipulation of the permission or thenon-permission using the display screen. For example, the selectionmanipulation of the permission or the non-permission can be receivedthrough a manipulation input using the manipulation member or amanipulation input based on a gesture.

Further, for example, the information processing device 630 causes thedisplay screen (for example, pop-up) for selecting the permission or thenon-permission to be displayed on a display unit 621 of another sourcedevice (for example, the information processing device 620). Then, theinformation processing device 630 can receive the selection manipulationof the permission or the non-permission performed on the display screen.

As described above, the permission or the non-permission can be set eachtime there is the connection request from the source device. Thus, forexample, even when confidentiality or a degree of importance isdifferent, the permission or the non-permission can be appropriatelyset.

In this example, the sink device functions as the group authenticationserver, and performs management of determining whether or not the sinkdevice gives permission for the outdoor use. However, a managementdevice (for example, a management server) that manages the permissionand the non-permission of the outdoor use in association with the sourcedevice may be newly installed, and the management of determining whetheror not permission for the outdoor use is given may be performed throughthe management device.

When the outdoor use information for giving the permission for theoutdoor use is included in the connection permission information (stepS844), the control unit of the information processing device 610 causesinformation indicating that the outdoor use is permitted to be displayedon the display unit 611 (step S845). For example, information indicatingthat the use within the same floor (the range that the radio waves ofthe access point 601 reach) and the outdoor use are permitted can bedisplayed in a pop-up manner, and the user can be notified of it.

Then, the control unit of the information processing device 610transmits the capability information related to the informationprocessing device 610 and public network service information to theinformation processing device 630 via the access point 601 (step S846).Here, the public network service information is information that enablesthe information processing device 610 to use a public network service,and includes, for example, a connection destination telephone number (atelephone number of the information processing device 610), anauthentication password, or the like.

Then, the control unit of the information processing device 610 performsa communication process (step S847). In the communication process, theprocess (steps S825 to S837) illustrated in FIG. 30 is performed.

Then, the control unit of the information processing device 610determines whether or not the user has moved to the outside of the radiowave transfer range of the access point 601 (step S848). Then, when theuser has not moved to the outside of the radio wave transfer range ofthe access point 601 (step S848), the process returns to step S847.

When the user has moved to the outside of the radio wave transfer rangeof the access point 601 (step S848), the control unit of the informationprocessing device 610 confirms whether or not the user desires theoutdoor use continuously (step S849). For example, the user may performan operation (for example, a communication end manipulation) ofdisconnecting the connection with the sink device. Thus, it is confirmedwhether or not the user desires the outdoor use continuously.

For example, the control unit of the information processing device 610causes a pop-up screen including information indicating that the userhas moved to the outside of the radio wave transfer range of the accesspoint 601 and a selection button for selecting whether or not the userdesires the outdoor use continuously to be displayed on the display unit611. Then, the control unit of the information processing device 610determines whether or not the selection button for selecting thecontinuous outdoor use has been pressed on the pop-up screen (stepS849). In other words, the control unit of the information processingdevice 610 determines whether or not the continuous use in a place aftermovement has been selected (step S849). When the continuous use in theplace after the movement has not been selected (step S849), theoperation of the data transmission process ends.

As described above, when the outdoor use information (permissioninformation) for giving the permission for the outdoor use has beenreceived, the control unit of the information processing device 610performs control such that the interchange with the informationprocessing device 630 via the access point other than the access point601 or the base station is performed. In this case, the control unit ofthe information processing device 610 can perform control such that atleast one of image data and audio data is received from the informationprocessing device 630 via the access point other than the access point601 or the base station and output.

In this example, the confirmation of the continuous outdoor use isperformed each time the movement to the outside of the radio wavetransfer range of the access point 601 is detected, but the presence orabsence of the continuous outdoor use may be set in advance. By settingthe presence or absence of the continuous outdoor use in advance asdescribed above, switching between the use within the same floor and theoutdoor use can be automatically performed seamlessly.

When the continuous use in the place after the movement is selected(step S849), the information processing device 630 performs theconnection process via the base station installed in the place after themovement (step S850). For example, the control unit of the informationprocessing device 630 determines whether or not the informationprocessing device 610 has moved to the outside of the radio wavetransfer range of the access point 601. Then, when the informationprocessing device 610 has moved to the outside of the radio wavetransfer range of the access point 601, the control unit of theinformation processing device 630 performs the connection process forestablishing the connection with the information processing device 610via the base station installed in the place after the movement. Forexample, when the telephone number is included in the public lineservice information received from the information processing device 610,the control unit of the information processing device 630 originates acall to the information processing device 610 via the public networkbased on the telephone number. Then, the control unit of the informationprocessing device 610 performs the connection process with theinformation processing device 630 via the public network.

Here, when the user has moved to the outside of the radio wave transferrange of the access point 601, the user owning the informationprocessing device 610 may not view the display unit 632 of the displaydevice 631 and audio output from the display device 631 may be hardlyaudible. In this regard, the information processing device 630 performsan interchange for confirming whether or not the information processingdevice 610 can receive an image and audio at a point in time at whichthe connection with the information processing device 610 is establishedvia the public network. Then, the information processing device 630determines whether the information processing device 610 can receiveboth or either of the image and the audio.

Based on the determination result, the information processing device 630performs the data communication (the image data and the audio data) withthe information processing device 610. In other words, the control unitof the information processing device 610 performs the data communication(the image data and the audio data) with the information processingdevice 630 via the public network. In the data communication, a two-wayinterchange is performed between the information processing device 610and the information processing device 630.

For example, the control unit of the information processing device 630transmits at least one of the display screen displayed on the displayunit 632 of the display device 631 and the audio output from the displaydevice 631. In the case of outdoor places, it may be difficult to viewthe display screen displayed on the display unit 611 of the informationprocessing device 610. In this case, only the audio may be transmitted.Data to be transmitted can be configured to be modified by the user ofthe information processing device 610.

Further, when the two-way data interchange is performed between theinformation processing device 610 and the information processing device630, the information processing device 630 may confirm a transmissionpath (for example, a band or a communication state) to the informationprocessing device 610 via the public network. Then, the informationprocessing device 630 may switch data to be transmitted based on theconfirmation result. For example, when a large-capacity transmissionpath is secured, the information processing device 630 transmits both ofthe image data and the audio data. Further, for example, when thelarge-capacity transmission path is not secured, the informationprocessing device 630 reduces an amount of image data by transcoding ortransmits only the audio data.

Then, the control unit of the information processing device 610determines whether or not the connection with the information processingdevice 630 via the public network is continuously used (step S852). Forexample, the control unit of the information processing device 610causes a display screen for selecting whether or not the connection withthe information processing device 630 via the public network iscontinuously used to be displayed (consistently or regularly) on thedisplay unit 611. Then, based on the selection manipulation in thedisplay screen, the control unit of the information processing device610 can determine whether or not the connection with the informationprocessing device 630 via the public network is continuously used.

When the connection with the information processing device 630 via thepublic network is continuously used (step S852), the process returns tostep S851. When the connection with the information processing device630 via the public network is not continuously used (step S852), thecontrol unit of the information processing device 610 disconnects theconnection with the information processing device 630 via the publicnetwork (step S853).

When the outdoor use information indicating that the outdoor use is notpermitted is included in the connection permission information (stepS844), the control unit of the information processing device 610 causesinformation indicating that the outdoor use is not permitted to bedisplayed on the display unit 611 (step S854).

Then, the control unit of the information processing device 610transmits the capability information related to the informationprocessing device 610 to the information processing device 630 via theaccess point 601 (step S855).

Then, the control unit of the information processing device 610 performsthe communication process (step S856). In the communication process, theprocess (steps S825 to S837) illustrated in FIG. 30 is performed.

Then, the control unit of the information processing device 610determines whether or not the user has moved to the outside of the radiowave transfer range of the access point 601 (step S857). Then, when theuser has not moved to the outside of the radio wave transfer range ofthe access point 601 (step S857), the process returns to step S856. Whenthe user has moved to the outside of the radio wave transfer range ofthe access point 601 (step S857), the process proceeds to step S853.

As described above, when the sink device and a plurality of sourcedevices share information, the information can be interchangedregardless of the position of the source device. For example, even whenthe student owning the information processing device 610 has moved tothe outside of the radio wave arrival range 605 of the access point 601,the information processing device 610 can be connected to theinformation processing device 630 via the base station. For example, thestudent may move from the classroom in which a presentation has beengiven to a laboratory or a place outside a university. Further, forexample, when the communication system 600 is installed in an office, asalesman may go out of a conference room in which a presentation hasbeen given. In this case, the salesman can listen to details of aconference outside (for example, in a train).

In the example illustrated in FIGS. 31 and 32, the management based onthe WebSocket is not limited to the access point and the Wi-Fi CERTIFIEDMiracast and can be performed even in switching with the public line.For example, when the public line is used, the management based on theWebSocket may be performed on the public line.

3. THIRD EMBODIMENT

In the second embodiment of the present technology, the example in whichthe source device is connected to the sink device or the access pointthrough the manipulation of the user or the operation of the user hasbeen described.

Here, the source device may be connected to the access point or the sinkdevice using the wireless LAN, and then the connection may bedisconnected. In this case, the connection setting of the wireless LANmay be stored in the source device even after the connection isdisconnected. In this case, the source device may be automaticallyconnected to the access point or the sink device even if the user owningthe source device does not perform a setting manipulation for connectionor the like.

In this regard, in the third embodiment of the present technology, anexample in which the source devices are automatically connected to theaccess point or the sink device will be described. In an environment inwhich the source device is automatically connected to the access pointor the sink device, there may be a plurality of access points connectedto various kinds of sink devices. Thus, by continuously maintaining theconnection of the source device to the plurality of access pointsthrough soft handover, such an environment is implemented.

First, an example in which the source device is automatically connectedto the access point will be described. For example, a case in which itis automatically connected to a private wireless LAN continuously isassumed.

[Operation Example of Information Processing Device (Source Device)]

FIG. 33 is a flowchart illustrating an example of a processing procedureof a data transmission process performed by the information processingdevice 610 according to the third embodiment of the present technology.The example illustrated in FIG. 33 is a modified example of FIG. 30,parts that are common to FIG. 30 are denoted by the same referencenumerals, and thus a description thereof is partially omitted.

First, the information processing device 610 is automatically handedover and connected to the access point installed in the place in whichthe information processing device 610 is located (step S861). Forexample, the user owning the information processing device 610 may moveto a specific floor to participate in the group. In this environment,the information processing device 610 activates an application (stepS862). After the activation, the control unit of the informationprocessing device 610 attempts the device discovery process for a sinkdevice that is desired to be connected.

A plurality of sink devices corresponding to the activated applicationmay be discovered through the device discovery process. In this case,the selection screen for selecting a desired sink device from aplurality of discovered sink devices may be displayed on the displayunit 611 of the information processing device 610, and the desired sinkdevice may be selected on the selection screen by the manipulation ofthe user (step S863). A device closest to the source device among aplurality of discovered sink devices may be automatically selected andconnected (step S863).

Next, an example in which the source device is automatically connectedto the sink device and access point will be described. For example, theconnection setting of the sink device is assumed to be stored in thesource device, and the source device is assumed to move into the radiowave arrival range of the sink device and be automatically connected tothe sink device.

[Operation Example of Information Processing Device (Source Device)]

FIG. 34 is a flowchart illustrating an example of a processing procedureof the data transmission process performed by the information processingdevice 610 according to the third embodiment of the present technology.The example illustrated in FIG. 34 is a modified example of FIG. 30,parts that are common to FIG. 30 are denoted by the same referencenumerals, and thus a description thereof is partially omitted.

First, the control unit of the information processing device 610determines whether or not the information processing device 610 has aconcurrent function (a time division concurrent function or asimultaneous use concurrent function) as a wireless LAN function (stepS871). When the information processing device 610 has the concurrentfunction, the information processing device 610 can establish the timedivision connection or the simultaneous connection with both of theaccess point and the sink device without performing disconnectionswitching of the access point and the sink device.

When the information processing device 610 has the concurrent function(step S871), the control unit of the information processing device 610performs a setting (a setting up to acquisition of an IP address) up toa layer 3 through the automatic connection (step S872). Then, thecontrol unit of the information processing device 610 makes an attemptto establish the connection with the sink device via the access point(step S873). Here, when the connection with the sink device isestablished based on both links of the connection via the access pointand the direct P2P connection, it is desirable that the device discoveryprocess be performed through only one of the connection via the accesspoint and the direct P2P connection. For example, a priority may begiven to the connection via the access point, only link connection maybe maintained for the direct P2P connection, and the image transmissionaccording to the Wi-Fi CERTIFIED Miracast may be stopped. In this case,in the image transmission mode, a high-speed operation can be performedduring a next switching period of time without wasting the frequencyband used in the Wi-Fi CERTIFIED Miracast.

When the information processing device 610 has no concurrent function(step S871), the control unit of the information processing device 610confirms the connection state with the sink device (step S874). Here,for example, when the connection with the sink device is the direct P2Pconnection, the direct P2P connection is first disconnected, andswitching to the connection via the access point is performed. Further,when the connection with the sink device is the connection via theaccess point, the connection is maintained. Further, when theinformation processing device 610 has no concurrent function, the portinformation and the IP information of the connection state are retainedin the information processing device 710 and diverted.

Then, the control unit of the information processing device 610 performsthe device discovery process via the access point (step S875). After thedevice discovery process, the control unit of the information processingdevice 610 determines whether or not the connection permissioninformation has been received from the sink device via the access point(step S876).

Here, in an environment in which the number of participating devices issmall, direct P2P connection may be maintained without change. In thiscase, when the connection permission information has been received fromthe sink device via the access point (step S876), the process proceedsto step S828.

As described above, it is possible to appropriately detect the currentsetting information without the user worrying about the connectionsetting. Further, it is possible to perform association so that the sameoperation as in the second embodiment of the present technology isperformed.

As described above, even in the device connection management, since theline via the access point is consistently connected, it is unnecessaryto switch the management based on the WebSocket between the access pointand the Wi-Fi CERTIFIED Miracast. Thus, it is possible to consistentlyperform the management through the link via the access point.

4. FOURTH EMBODIMENT

The first to third embodiments of the present technology have beendescribed in connection with the example in which a plurality of sourcedevices are connected with one sink device. However, the first to thirdembodiments of the present technology can be applied even when onesource device is connected with a plurality of sink devices. Further,the first to third embodiments of the present technology can be appliedeven when a plurality of source devices are connected with a pluralityof sink devices.

In this regard, the fourth embodiment of the present technology will bedescribed in connection with an example in which one source device isconnected with a plurality of sink devices.

[Configuration Example of Communication System]

FIG. 35 is a diagram illustrating a system configuration example of acommunication system 700 according to a fourth embodiment of the presenttechnology.

The communication system 700 includes the Internet 701, base stations702 and 703, imaging devices 704 and 705, a receiver 710, andinformation processing devices 720 and 730.

The base stations 702 and 703 are a base station that enables thereceiver 710 and the information processing devices 720 and 730 to beconnected to the Internet 701 using wireless communication.

The imaging devices 704 and 705 are an imaging device that photographs asubject and generates image data. The imaging devices 704 and 705 cantransmit the generated image data to the receiver 710 and theinformation processing devices 720 and 730 via the Internet 701.

The receiver 710 is a television receiver (for example, a televisionreceiver with a video recording function) that receives a broadcast wavefrom each broadcast station and displays an image (a stereoscopic imageor a planar image). The receiver 710 is assumed to be an informationprocessing device capable of providing a display environment in which abroadcast and communication are fused as in a hybridcast.

The receiver 710 can transmit various kinds of images (for example, animage input from an external device, an image based on a broadcast wave,an image stored in an internal storage unit, and an image displayed onthe display unit 711) to the information processing devices 720 and 730using wireless communication.

The receiver 710 is assumed to be the transmission side informationprocessing device (the source device), and the information processingdevices 720 and 730 are assumed to be the reception side informationprocessing device (the sink device). The receiver 710 is assumed tocorrespond to the information processing device 200 illustrated in FIG.2, and the information processing devices 720 and 730 are assumed tocorrespond to the information processing device 300 illustrated in FIG.3.

For example, the information processing device 720 can receive imagedata (for example, image data obtained by photographing a landscape)transmitted from the imaging devices 704 and 705 via the base station702 and cause an image based on each piece of image data to be displayedon the display unit 721. Similarly, the information processing device730 can receive image data (for example, image data obtained byphotographing a landscape) transmitted from the imaging devices 704 and705 via the base station 702 and cause an image based on each piece ofimage data to be displayed on the display unit 731.

The receiver 710 can cause the image based on the broadcast wavereceived from each broadcast station to be displayed on the display unit711. The receiver 710 can transmit image data based on the broadcastwave received from each broadcast station to the information processingdevices 720 and 730. In this case, the information processing devices720 and 730 can cause the image based on the broadcast wave to bedisplayed on the display units 721 and 731, respectively.

The image based on the broadcast wave may not be displayed on the sourcedevice and the sink device at the same time due to a copyright issue. Inthis regard, when the source device (the receiver 710) transmits theimage data based on the broadcast wave to be displayed on the sinkdevice (the information processing devices 720 and 730), the image iserased from the source device before the transmission. Another image isprevented from being overwritten on the image (the image based on thebroadcast wave) displayed on the sink device.

The image based on the broadcast wave may be an image corresponding to ahigh-bandwidth digital content protection system (HDCP). For thisreason, when the image (the HDCP use image) based on the broadcast waveis displayed on the display unit 721, the information processing device720 causes another image not to overlap the image based on the broadcastwave. For example, the information processing device 720 causes theimages transmitted from the imaging devices 704 and 705 not to beoverwritten on the image (the HDCP use image) based on the broadcastwave and displayed.

Further, when an image transmitted from one source device is displayedon a plurality of sink devices, similarly to the first to thirdembodiments of the present technology, it is possible to notify the sinkdevices of the display position of the source device in real time andcontrol the display position from the sink device. For example, when theimage transmitted from the receiver 710 is displayed on the informationprocessing devices 720 and 730, it is possible to notify the informationprocessing devices 720 and 730 of the display position of the receiver710 and control the display position from the information processingdevices 720 and 730.

For example, the information processing devices 720 and 730 can set adesired image and acquire the image from the receiver 710. For example,the information processing device 720 is assumed to set keywordinformation (for example, a Japan national football team). In this case,images related to the keyword information (for example, the Japannational football team) are extracted in the receiver 710, and theextracted images are transmitted from the receiver 710 to theinformation processing device 720. Then, the images related to thekeyword information (for example, the Japan national football team) canbe displayed on the display unit 721 of the information processingdevice 720.

[Operation Example when Non-Display Region is Notified of]

As described above, another image is prevented from overlapping the HDCPuse image. In this regard, an example in which a region (a non-displayregion) that does not overlap another image is decided and reported toanother device will be described.

First, the description will proceed with an operation example of thesource device in the multi-sink environment.

FIG. 36 is a flowchart illustrating an example of a processing procedureof a non-display region notification process performed by the receiver710 (the source device) according to the fourth embodiment of thepresent technology.

First, a control unit of the receiver 710 detects whether or not thesink device that newly participates in has a capability capable ofdecoding the HDCP at the time of the Capability Negotiation of the sinkdevice connected to the receiver 710 as connection information (stepS881).

Then, the control unit of the receiver 710 performs control such thatanother image except the image of the receiver 710 is not overwrittenwhen the sink device that newly participates in decodes and displays theHDCP at the time of the Capability Negotiation (step S882).

Then, the control unit of the receiver 710 determines whether or not theHDCP is used for the connection between the receiver 710 and the sinkdevice based on the capability information (step S883). When the HDCP isnot used for the connection between the receiver 710 and the sink device(step S883), the process returns to step S881.

When the HDCP is used for the connection between the receiver 710 andthe sink device (step S883), the control unit of the receiver 710 decidethe region (the non-display region) in which the images received from aplurality of source devices are prevented from being displayed tooverlap (step S884).

Then, the control unit of the receiver 710 determines whether or notthere is another sink device connected to the receiver 710 (step S885).When there is another sink device connected to the receiver 710 (stepS885), the process returns to step S881.

When there is another sink device connected to the receiver 710 (stepS885), the control unit of the receiver 710 notifies the sink devices ofthe decided non-display region (step S886).

Next, an operation example of the sink device in the multi-sourceenvironment will be described. This operation example will be alsodescribed with reference to FIG. 36.

First, a control unit of the information processing device 720 detectsthe source device that newly participates in uses the HDCP at the timeof the Capability Negotiation of the source device connected to theinformation processing device 720 as the connection information (stepS881).

Then, the control unit of the information processing device 720 displaysthe image without overlapping when the source device that newlyparticipates in uses the HDCP at the time of the Capability Negotiationof the source device connected to the information processing device 720(step S882).

Then, the control unit of the information processing device 720determines whether or not the HDCP is used for the connection betweenthe information processing device 720 and the source device based on thecapability information (step S883). When the HDCP is not used for theconnection between the information processing device 720 and the sourcedevice (step S883), the process returns to step S881.

When the HDCP is used for the connection between the informationprocessing device 720 and the source device (step S883), the controlunit of the information processing device 720 decides the region (thenon-display region) in which the images received from a plurality ofsource devices are prevented from being displayed to overlap (stepS884).

Then, the control unit of the information processing device 720determines whether or not there is another source device connected tothe information processing device 720 (step S885). When there is anothersource device connected to the information processing device 720 (stepS885), the process returns to step S881.

When there is no another source device connected to the informationprocessing device 720 (step S885), the control unit of the informationprocessing device 720 notifies the source devices of the decidednon-display region (step S886).

As described above, when the image transmitted from one source deviceare displayed on a plurality of sink devices, it is possible to notifythe sink devices of the display position of the source device in realtime. Thus, it is possible to control the display position from the sinkdevice.

5. APPLICATION EXAMPLES

The technology of the present disclosure can be applied to variousproducts. For example, the information processing device 200, 300, 400,603, 604, 610, 620, 630, 720, 730, and the receiver 710 may be realizedas a mobile terminal such as a smartphone, a tablet-type personalcomputer (PC), a notebook PC, a portable game terminal, or a digitalcamera, a fixed-type terminal such as a television receiver set, aprinter, a digital scanner, or a network storage, or an in-vehicleterminal such as a car navigation device. In addition, the informationprocessing device 200, 300, 400, 603, 604, 610, 620, 630, 720, 730, andthe receiver 710 may be realized as a terminal which performsmachine-to-machine (M2M) communication (which is also referred to as amachine-type communication (MTC) terminal) such as a smart meter, avending machine, a remote monitoring device, or a point-of-sale (POS)terminal. Furthermore, the information processing device 200, 300, 400,603, 604, 610, 620, 630, 720, 730, and the receiver 710 may be awireless communication module (for example, an integrated circuit moduleconfigured in one die) mounted in these terminals.

5-1. First Application Example

FIG. 37 is a block diagram showing an example of a schematicconfiguration of a smartphone 900 to which the technology of the presentdisclosure may be applied. The smartphone 900 includes a processor 901,a memory 902, a storage 903, an external connection interface 904, acamera 906, a sensor 907, a microphone 908, an input device 909, adisplay device 910, a speaker 911, a wireless communication interface913, an antenna switch 914, an antenna 915, a bus 917, a battery 918,and an auxiliary controller 919.

The processor 901 may be, for example, a central processing unit (CPU)or a system on a chip (SoC), and controls functions of an applicationlayer and another layer of the smartphone 900. The memory 902 includes arandom access memory (RAM) and a read only memory (ROM), and stores aprogram that is executed by the processor 901, and data. The storage 903may include a storage medium such as a semiconductor memory or a harddisk. The external connection interface 904 is an interface forconnecting an external device such as a memory card or a universalserial bus (USB) device to the smartphone 900.

The camera 906 includes an image sensor such as a charge coupled device(CCD) and a complementary metal oxide semiconductor (CMOS), andgenerates a captured image. The sensor 907 may include a group ofsensors such as a measurement sensor, a gyro sensor, a geomagneticsensor, and an acceleration sensor. The microphone 908 converts soundsthat are input to the smartphone 900 to audio signals. The input device909 includes, for example, a touch sensor configured to detect touchonto a screen of the display device 910, a keypad, a keyboard, a button,or a switch, and receives an operation or an information input from auser. The display device 910 includes a screen such as a liquid crystaldisplay (LCD) and an organic light-emitting diode (OLED) display, anddisplays an output image of the smartphone 900. The speaker 911 convertsaudio signals that are output from the smartphone 900 to sounds.

The wireless communication interface 913 supports one or more ofwireless LAN standards such as IEEE 802.11a, 11b, 11g, 11n, 11ac, and11ad to execute wireless communication. The wireless communicationinterface 913 can communicate with another device via a wireless LANaccess point in an infrastructure mode. In addition, the wirelesscommunication interface 913 can directly communicate with another devicein an ad hoc mode or a direct communication mode such as Wi-Fi direct,or the like. In the Wi-Fi Direct, unlike the ad-hoc mode, one of twoterminals operates as an access point, but communication is performeddirectly between the terminals. The wireless communication interface 913can typically include a baseband processor, a radio frequency (RF)circuit, and a power amplifier. The wireless communication interface 913may be a one-chip module in which a memory which stores a communicationcontrol program, a processor which executes the program and a relevantcircuit are integrated. The wireless communication interface 913 maysupport other kinds of wireless communication schemes such as a nearfield wireless communication scheme, a proximity wireless communicationscheme or a cellular communication scheme in addition to the wirelessLAN scheme. The antenna switch 914 switches connection destinations ofthe antenna 915 between a plurality of circuits (for example, circuitsfor different wireless communication schemes) included in the wirelesscommunication interface 913. The antenna 915 has a single or a pluralityof antenna elements (for example, a plurality of antenna elements whichconstitute a MIMO antenna), which are used by the wireless communicationinterface 913 for transmission and reception of radio signals. Awireless communication interface function for establishing a connectionwith the public line that conforms to IEEE 802.16, the 3GPPspecifications (for example, W-CDMA, GSM, WiMAX, WiMAX2, LTE, andLTE-A), or the like may be provided so that communication with thepublic line is performed.

It should be noted that the smartphone 900 is not limited to the exampleof FIG. 37 and may include a plurality of antennas (for example, anantenna for a wireless LAN, an antenna for the proximity wirelesscommunication scheme, an antenna for the public line communication,etc.). In that case, the antenna switch 914 may be omitted from theconfiguration of the smartphone 900.

The bus 917 connects the processor 901, the memory 902, the storage 903,the external connection interface 904, the camera 906, the sensor 907,the microphone 908, the input device 909, the display device 910, thespeaker 911, the wireless communication interface 913, and the auxiliarycontroller 919 to each other. The battery 918 supplies power to blocksof the smartphone 900 illustrated in FIG. 37 via feeder lines, which arepartially shown as dashed lines in the figure. The auxiliary controller919 operates a minimum necessary function of the smartphone 900, forexample, in a sleep mode.

In the smartphone 900 shown in FIG. 37, the control unit 240, and thecontrol unit 370 described using FIG. 3 may be implemented by thewireless communication interface 913. In addition, at least some ofthese functions may be implemented by the processor 901 or the auxiliarycontroller 919.

The smartphone 900 may operate as a wireless access point (a softwareAP) by executing an access point function at an application levelthrough the processor 901. The wireless communication interface 913 mayhave a wireless access point function.

5-2 Second Application Example

FIG. 38 is a block diagram illustrating an example of a schematicconfiguration of a car navigation device 920 to which the technology ofthe present disclosure may be applied. The car navigation device 920includes a processor 921, a memory 922, a global positioning system(GPS) module 924, a sensor 925, a data interface 926, a content player927, a storage medium interface 928, an input device 929, a displaydevice 930, a speaker 931, a wireless communication interface 933, anantenna switch 934, an antenna 935, and a battery 938.

The processor 921 may be, for example, a CPU or a SoC, and controls anavigation function and another function of the car navigation device920. The memory 922 includes RAM and ROM, and stores a program that isexecuted by the processor 921, and data.

The GPS module 924 uses GPS signals received from a GPS satellite tomeasure a position (such as latitude, longitude, and altitude) of thecar navigation device 920. The sensor 925 may include a group of sensorssuch as a gyro sensor, a geomagnetic sensor, and an air pressure sensor.The data interface 926 is connected to, for example, an in-vehiclenetwork 941 via a terminal that is not shown, and acquires datagenerated by the vehicle, such as vehicle speed data.

The content player 927 reproduces content stored in a storage medium(such as a CD and a DVD) that is inserted into the storage mediuminterface 928. The input device 929 includes, for example, a touchsensor configured to detect touch onto a screen of the display device930, a button, or a switch, and receives an operation or an informationinput from a user. The display device 930 includes a screen such as aLCD or an OLED display, and displays an image of the navigation functionor content that is reproduced. The speaker 931 outputs sounds of thenavigation function or the content that is reproduced.

The wireless communication interface 933 supports one or more ofwireless LAN standards such as IEEE 802.11a, 11b, 11g, 11n, 11ac, and11ad to execute wireless communication. The wireless communicationinterface 933 can communicate with another device via a wireless LANaccess point in an infrastructure mode. In addition, the wirelesscommunication interface 933 can directly communicate with another devicein an ad hoc mode or a direct communication mode such as Wi-Fi Direct.The wireless communication interface 933 can typically include abaseband processor, an RF circuit, and a power amplifier. The wirelesscommunication interface 933 may be a one-chip module in which a memorywhich stores a communication control program, a processor which executesthe program and a relevant circuit are integrated. The wirelesscommunication interface 933 may support other kinds of wirelesscommunication schemes such as a near field wireless communicationscheme, a proximity wireless communication scheme or a cellularcommunication scheme in addition to the wireless LAN scheme. The antennaswitch 934 switches connection destinations of the antenna 935 between aplurality of circuits included in the wireless communication interface933. The antenna 935 has a single or a plurality of antenna elements,which are used by the wireless communication interface 933 fortransmission and reception of radio signals.

In addition, the car navigation device 920 may include a plurality ofantennas, not limited to the example of FIG. 38. In that case, theantenna switches 934 may be omitted from the configuration of the carnavigation device 920.

The battery 938 supplies power to blocks of the car navigation device920 illustrated in FIG. 38 via feeder lines that are partially shown asdashed lines in the figure. The battery 938 accumulates power suppliedform the vehicle.

In the car navigation device 920 illustrated in FIG. 38, the controlunit 240 described using FIG. 2 and the control unit 370 described usingFIG. 3 may be implemented by the wireless communication interface 933.At least a part of the functions may also be implemented by theprocessor 921.

The technology of the present disclosure may also be realized as anin-vehicle system (or a vehicle) 940 including one or more blocks of thecar navigation device 920, the in-vehicle network 941, and a vehiclemodule 942. The vehicle module 942 generates vehicle data such asvehicle speed, engine speed, and trouble information, and outputs thegenerated data to the in-vehicle network 941.

The above-described embodiments are examples for embodying the presenttechnology, and matters in the embodiments each have a correspondingrelationship with disclosure-specific matters in the claims. Likewise,the matters in the embodiments and the disclosure-specific matters inthe claims denoted by the same names have a corresponding relationshipwith each other. However, the present technology is not limited to theembodiments, and various modifications of the embodiments may beembodied in the scope of the present technology without departing fromthe spirit of the present technology.

The processing sequences that are described in the embodiments describedabove may be handled as a method having a series of sequences or may behandled as a program for causing a computer to execute the series ofsequences and recording medium storing the program. As the recordingmedium, a hard disk, a CD (Compact Disc), an MD (MiniDisc), and a DVD(Digital Versatile Disc), a memory card, and a Blu-ray disc (registeredtrademark) can be used.

Effects described in the present description are just examples, theeffects are not limited, and there may be other effects.

Additionally, the present technology may also be configured as below.

(1)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with at least one sink device according to a wirelessfidelity (Wi-Fi) CERTIFIED Miracast specification; and

a control unit configured to perform control related to the real-timeimage transmission based on control information for controlling theinformation processing device according to the Wi-Fi CERTIFIED Miracastspecification, the control information being received from the at leastone sink device,

wherein the control unit performs control such that on-off switching ofa user input back channel (UIBC) for transmitting manipulationinformation in the sink device that has transmitted the controlinformation is performed when a plurality of pieces of controlinformation are received.

(2)

The information processing device according to (1),

wherein the wireless communication unit receives the control informationon a transmission control protocol (TCP) session generated between theinformation processing device and the sink device.

(3)

The information processing device according to (1) or (2),

wherein the control of the on-off switching is control of any one of aconnection, a disconnection, a start, a stop, validation, andinvalidation.

(4)

The information processing device according to (3),

wherein the control unit performs control such that the UIBC isvalidated for a first sink device among the at least one sink device,and the UIBC is invalidated for a second sink device.

(5)

The information processing device according to (4),

wherein the wireless communication unit gives a notification indicatingthat the UIBC is set to be valid to the first sink device, and gives anotification indicating that the UIBC is set to be invalid to the secondsink device.

(6)

The information processing device according to (5),

wherein the control information is a GET_PARAMETER Response, and thenotification is transmitted as SET_PARAMETER.

(7)

The information processing device according to (3),

wherein the control unit performs control such that the UIBC isvalidated for a first sink device among the at least one sink device,and the UIBC is invalidated for all the sink devices except the firstsink device among the sink devices that have transmitted the controlinformation.

(8)

The information processing device according to any of (1) to (7),

wherein the wireless communication unit receives a plurality of piecesof manipulation information associated with ID information as UIBCinformation, and

the control unit identifies corresponding manipulation information withreference to an ID header included in the ID information.

(9)

The information processing device according to claim 8,

wherein the at least one sink device includes a plurality of inputdevices, and

the plurality of pieces of manipulation information are transmitted fromthe plurality of input devices via the at least one sink device.

The information processing device according to (8).(10)

The information processing device according to (8) or (9),

wherein the wireless communication unit sets one port for a bundle ofthe plurality of pieces of UIBC information, and performs communication.

(11)

The information processing device according to any of (1) to (10),

wherein the wireless communication unit performs communication as asource device.

(12)

An information processing method, including:

a wireless communication process of performing real-time imagetransmission with at least one sink device according to a wirelessfidelity (Wi-Fi) CERTIFIED Miracast specification; and

a control process of performing control related to the real-time imagetransmission based on control information for controlling theinformation processing device according to the Wi-Fi CERTIFIED Miracastspecification, the control information being received from the at leastone sink device,

wherein the control process includes performing control such that on-offswitching of a user input back channel (UIBC) for transmittingmanipulation information in the sink device that has transmitted thecontrol information is performed when a plurality of pieces of controlinformation are received.

(13)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with at least one source device according to a wirelessfidelity (Wi-Fi) CERTIFIED Miracast specification; and

a display unit configured to display an image received through thereal-time image transmission,

wherein the wireless communication unit receives control informationrelated to turning-on or off of a user input back channel (UIBC) fortransmitting manipulation information in the information processingdevice among at least one sink device from a first source device whenthe first source device among the at least one source device performsthe real-time image transmission with the at least one sink device.

Additionally, the present technology may also be configured as below.

(C1)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with another information processing device according to awireless fidelity (Wi-Fi) CERTIFIED Miracast specification; and

a control unit configured to perform control such that an image based onimage information transmitted from the other information processingdevice is displayed on a display unit, and a port used for a pluralityof communication schemes for interchanging control information relatedto the image with the other information processing device is set.

(C2)

The information processing device according to (C1),

wherein the control unit transmits information designating the pluralityof communication schemes using the port to the other informationprocessing device when the port is set on a transmission controlprotocol (TCP) session generated between the information processingdevice and the other information processing device.

(C3)

The information processing device according to (C2),

wherein the control unit transmits information designating a user inputback channel (UIBC) and a WebSocket as the plurality of communicationschemes to the other information processing device.

(C4)

The information processing device according to (C3),

wherein the control unit acquires image information and audioinformation transmitted from the other information processing deviceusing the Web Socket, causes an image based on the image information tobe displayed on the display unit, and causes an audio based on the audioinformation to be output from an audio output unit.

(C5)

The information processing device according to any one of (C1) to (C4),

wherein the control unit sets a plurality of ports for the plurality ofcommunication schemes as the port used for a plurality of communicationschemes.

(C6)

The information processing device according to (C5),

wherein the control unit sets the plurality of ports according to arequest transmitted from the other information processing device.

(C7)

The information processing device according to any one of (C1) to (C4),

wherein the control unit sets one port for the plurality ofcommunication schemes as the port used for a plurality of communicationschemes.

(C8)

The information processing device according to any one of (C1) to (C7),

wherein the control unit transmits information related to a display formof the image in the display unit to the other information processingdevice using any one of the plurality of communication schemes, andchanges the display form of the image based on control informationtransmitted from the other information processing device.

(C9)

The information processing device according to (C8),

wherein the control information is manipulation information related to amanipulation for changing the display form of the image received in theother information processing device.

(C10)

The information processing device according to any one of (C1) to (C9),

wherein the control unit causes the image and information indicating theother information processing device to be displayed on the display unitin association with each other.

(C11)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with another information processing device according to awireless fidelity (Wi-Fi) CERTIFIED Miracast specification; and

a control unit configured to perform control such that image informationfor causing an image to be displayed on a display unit with which theother information processing device is equipped or a display unitconnected to the other information processing device is transmitted tothe other information processing device, and a port used for a pluralityof communication schemes for interchanging control information relatedto the image with the other information processing device is set.

(C12)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with a plurality of information processing devicesaccording to a wireless fidelity (Wi-Fi) CERTIFIED Miracastspecification; and

a control unit configured to perform control such that an image based onimage information transmitted from the plurality of informationprocessing devices is displayed on a display unit, and when a pluralityof ports for interchanging control information related to the image areset for the plurality of information processing devices, an informationprocessing device that interchanges the control information is selectedamong the plurality of information processing devices, and switching ofthe port for interchanging the control information with the selectedinformation processing device is performed.

(C13)

An information processing device, including:

a wireless communication unit configured to perform real-time imagetransmission with a plurality of information processing devicesaccording to a wireless fidelity (Wi-Fi) CERTIFIED Miracastspecification; and

a control unit configured to perform control such that image informationfor causing an image to be displayed on display units with which theplurality of information processing devices are equipped or displayunits connected to the plurality of information processing devices istransmitted to the plurality of information processing device, and whena plurality of ports for interchanging control information related tothe image are set for the plurality of information processing devices,an information processing device that interchanges the controlinformation is selected among the plurality of information processingdevices, and switching of the port for interchanging the controlinformation with the selected information processing device isperformed.

(C14)

An information processing device, including:

a wireless communication unit configured to perform data transmissionwith a plurality of other information processing devices; and

a control unit configured to receive control information for controllingthe information processing device from the plurality of otherinformation processing devices and select the control informationtransmitted from an appropriate device.

(C15)

The information processing device according to (C14),

wherein the control unit receives the control information on atransmission control protocol (TCP) session generated between theinformation processing device and the other information processingdevice.

(C16)

A communication system, including:

first and second information processing devices configured to performreal-time image transmission according to a wireless fidelity (Wi-Fi)CERTIFIED Miracast specification,

wherein the first information processing device transmits imageinformation to the second information processing device, and

the second information processing device includes a control unitconfigured to perform control such that an image based on the imageinformation transmitted from the first information processing device isdisplayed on a display unit, and a port used for a plurality ofcommunication schemes for interchanging control information related tothe image with the first information processing device is set.

(C17)

An information processing method, including:

a display process of causing an image based on image informationtransmitted from another information processing device to be displayedon a display unit according to a wireless fidelity (Wi-Fi) CERTIFIEDMiracast specification; and

a control process of setting a port used for a plurality ofcommunication schemes for interchanging control information related tothe image with the other information processing device.

(C18)

A program causing a computer to execute:

a display process of causing an image based on image informationtransmitted from another information processing device to be displayedon a display unit according to a wireless fidelity (Wi-Fi) CERTIFIEDMiracast specification; and

a control process of setting a port used for a plurality ofcommunication schemes for interchanging control information related tothe image with the other information processing device.

REFERENCE SIGNS LIST

-   100 communication system-   200 information processing device-   201 data transmission system-   202 line control system-   203 information processing device-   210 antenna-   220 wireless communication unit-   230 control signal reception unit-   240 control unit-   250 image and audio signal generation unit-   260 image and audio compression unit-   270 stream transmission unit-   300 information processing device-   301 the line control system-   302 input and output system-   310 antenna-   320 wireless communication unit-   330 stream reception unit-   340 image and audio decompression unit-   350 image and audio output unit-   351 display unit-   352 audio output unit-   360 user information acquisition unit-   361 imaging unit-   370 control unit-   380 control signal transmission unit-   390 management information retention unit-   400 information processing device-   410 source device-   420 sink device-   600 communication system-   601 access point-   602 network-   603, 604, 610, 620, 630 information processing device-   611 display unit-   621 display unit-   631 display device-   632 display unit-   640 control device-   643 input and output unit-   700 communication system-   701 the Internet-   702, 703 base station-   704, 705 imaging device-   710 receiver-   711 display unit-   720, 730 information processing device-   721, 731 display unit-   900 smartphone-   901 processor-   902 memory-   903 storage-   904 external connection interface-   906 camera-   907 sensor-   908 microphone-   909 input device-   910 display device-   911 speaker-   913 wireless communication interface-   914 antenna switch-   915 antenna-   917 bus-   918 battery-   919 auxiliary controller-   920 car navigation device-   921 processor-   922 memory-   924 GPS module-   925 sensor-   926 data interface-   927 content player-   928 storage medium interface-   929 input device-   930 display device-   931 speaker-   933 wireless communication interface-   934 antenna switch-   935 antenna-   938 battery-   941 in-vehicle network-   942 vehicle module-   1100, 1300, 1501 to 1504 sink device-   1110, 1310, 1410, 1610 display control unit-   1120, 1320, 1420, 1620 mode request content switching unit-   1211 to 1214, 1400, 1600 source device-   1330, 1630 common input switching unit

1. An information processing device, comprising: a wirelesscommunication unit configured to perform real-time image transmissionwith at least one sink device according to a wireless fidelity (Wi-Fi)CERTIFIED Miracast specification; and a control unit configured toperform control related to the real-time image transmission based oncontrol information for controlling the information processing deviceaccording to the Wi-Fi CERTIFIED Miracast specification, the controlinformation being received from the at least one sink device, whereinthe control unit performs control such that on-off switching of a userinput back channel (UIBC) for transmitting manipulation information inthe sink device that has transmitted the control information isperformed when a plurality of pieces of control information arereceived.
 2. The information processing device according to claim 1,wherein the wireless communication unit receives the control informationon a transmission control protocol (TCP) session generated between theinformation processing device and the sink device.
 3. The informationprocessing device according to claim 1, wherein the control of theon-off switching is control of any one of a connection, a disconnection,a start, a stop, validation, and invalidation.
 4. The informationprocessing device according to claim 3, wherein the control unitperforms control such that the UIBC is validated for a first sink deviceamong the at least one sink device, and the UIBC is invalidated for asecond sink device.
 5. The information processing device according toclaim 4, wherein the wireless communication unit gives a notificationindicating that the UIBC is set to be valid to the first sink device,and gives a notification indicating that the UIBC is set to be invalidto the second sink device.
 6. The information processing deviceaccording to claim 5, wherein the control information is a GET_PARAMETERResponse, and the notification is transmitted as SET_PARAMETER.
 7. Theinformation processing device according to claim 3, wherein the controlunit performs control such that the UIBC is validated for a first sinkdevice among the at least one sink device, and the UIBC is invalidatedfor all the sink devices except the first sink device among the sinkdevices that have transmitted the control information.
 8. Theinformation processing device according to claim 1, wherein the wirelesscommunication unit receives a plurality of pieces of manipulationinformation associated with ID information as UIBC information, and thecontrol unit identifies corresponding manipulation information withreference to an ID header included in the ID information.
 9. Theinformation processing device according to claim 8, wherein the at leastone sink device includes a plurality of input devices, and the pluralityof pieces of manipulation information are transmitted from the pluralityof input devices via the at least one sink device.
 10. The informationprocessing device according to claim 8, wherein the wirelesscommunication unit sets one port for a bundle of the plurality of piecesof UIBC information, and performs communication.
 11. The informationprocessing device according to claim 1, wherein the wirelesscommunication unit performs communication as a source device.
 12. Aninformation processing method, comprising: a wireless communicationprocess of performing real-time image transmission with at least onesink device according to a wireless fidelity (Wi-Fi) CERTIFIED Miracastspecification; and a control process of performing control related tothe real-time image transmission based on control information forcontrolling the information processing device according to the Wi-FiCERTIFIED Miracast specification, the control information being receivedfrom the at least one sink device, wherein the control process includesperforming control such that on-off switching of a user input backchannel (UIBC) for transmitting manipulation information in the sinkdevice that has transmitted the control information is performed when aplurality of pieces of control information are received.
 13. Aninformation processing device, comprising: a wireless communication unitconfigured to perform real-time image transmission with at least onesource device according to a wireless fidelity (Wi-Fi) CERTIFIEDMiracast specification; and a display unit configured to display animage received through the real-time image transmission, wherein thewireless communication unit receives control information related toturning-on or off of a user input back channel (UIBC) for transmittingmanipulation information in the information processing device among atleast one sink device from a first source device when the first sourcedevice among the at least one source device performs the real-time imagetransmission with the at least one sink device.